Description

xml2db uses sqlalchemy insert statements to load data to the database, which is easy because it is backend and driver independant. However, performance is not great for larger datasets, even with fast_executemany enabled. It is rather impractical in production use for datasets over a few MB.

Most backends and drivers offer solutions to bulk load data to a table on a server, usually from a csv type format and bypassing transaction logs, which is much faster. It is quite easy to use them with xml2db by using a custom implementation of Document.insert_into_target_tables and more specifically Document.insert_into_temp_tables.

Proposed feature

This could be at least documented with some examples, or even implemented within the package, for some backends/drivers (at least the ones we are running tests against, which all support this kind of bulk loading).