Hi all.
Got the connector installed, the source side works, which tells me I got it correctly installed,
Trying to do a sink, and the connector is failing on the Confluent side.
I’m posting below both my source and sink as examples. Confluent logs are not very helpful,
this is a local Mongodb running in a container and a local deployed Confluent stack, the one outside of containers…
I’ve also tried posting directly to Kafka topic and sinking that into a collection, same error/result, sink fails.
G
curl -X POST \
-H "Content-Type: application/json" \
--data '
{"name": "mongo-source",
"config": {
"connector.class":"com.mongodb.kafka.connect.MongoSourceConnector",
"connection.uri":"mongodb://localhost:27017/?directConnection=true",
"database":"quickstart",
"collection":"sampleData",
"pipeline":"[{\"$match\": {\"operationType\": \"insert\"}}, {$addFields : {\"fullDocument.travel\":\"MongoDB Kafka Connector\"}}]"
}
}
' \
http://localhost:8083/connectors -w "\n"
curl -X POST \
-H "Content-Type: application/json" \
--data '
{"name": "mongo-sink",
"config": {
"connector.class":"com.mongodb.kafka.connect.MongoSinkConnector",
"connection.uri":"mongodb://localhost:27017/?directConnection=true",
"database":"quickstart",
"collection":"topicData",
"topics":"quickstart.sampleData",
"change.data.capture.handler": "com.mongodb.kafka.connect.sink.cdc.mongodb.ChangeStreamHandler"
}
}
' \
http://localhost:8083/connectors -w "\n"
Below is the sink connector that takes messages posted directly onto a kafka topic, (not sourced via a source connector, fails similar to above.
curl -X POST \
-H "Content-Type: application/json" \
--data '
{"name": "mongo-creator-sink",
"config": {
"connector.class":"com.mongodb.kafka.connect.MongoSinkConnector",
"connection.uri":"mongodb://localhost:27017/?directConnection=true",
"database":"MongoCom0",
"collection":"creator_salesbasket",
"topics":"mongocreator_basket",
"change.data.capture.handler": "com.mongodb.kafka.connect.sink.cdc.mongodb.ChangeStreamHandler"
}
}
' \
http://localhost:8083/connectors -w "\n"
`