-
Notifications
You must be signed in to change notification settings - Fork 479
FAQ
This document encompasses many of the frequently asked questions (FAQs) about Mongo Connector.
Mongo Connector updates the oplog progress file (called config.txt
, by default) whenever its cursor into the MongoDB oplog is closed. Note that this may come long after Mongo Connector has read and processed all entries currently in the oplog. This is due to the connector's use of a tailable cursor, which can be re-used to retrieve documents that arrive in the oplog even after the cursor is created. Thus, you cannot rely on the progress file being updated automatically after the oplog is exhausted.
Instead, Mongo Connector provides the --batch-size
option with which you can specify the maximum number of documents Mongo Connector may process before having to record its progress. For example, if you wanted to make sure that Mongo Connector records its progress at least every 100 operations in the oplog, you could run:
mongo-connector -m <source host/port> -t <destination host/port> --batch-size=100
Documents that are missing or have additional fields to the Solr collection schema cannot be inserted, and Solr will log an exception. Thus, Mongo Connector tries to read your Solr collection's schema prior to replicating any operations to Solr in order to avoid sending invalid requests. Documents replicated to Solr from MongoDB may need to be altered to remove fields that aren't in the schema, and the result may look as if your documents are missing certain fields.
The solution to this is to update your schema.xml
file and [reload])https://wiki.apache.org/solr/CoreAdmin#RELOAD) the relevant Solr cores.