Transform timestamp in mongo-kafka connector

Sendoh_Daten · 2020-02-17T12:16:07.133Z

When using mongo-kafka source connector, I receive

Caused by: org.apache.kafka.con │ │ nect.errors.DataException: Only Struct objects supported for [converting timestamp formats], found: java.lang.String\n\tat org.apache.kafka.connect.transforms.util.Requirements.requireStruct(Requireme │ │ nts.java:52)\n\tat org.apache.kafka.connect.transforms.util.Requirements.requireStructOrNull(Requirements.java:61)

with the following config for transforms:

"transforms": "revecied_at_unix_to_iso",
"transforms.revecied_at_unix_to_iso.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value",
"transforms.revecied_at_unix_to_iso.field": "fullDocument.received.$date",
"transforms.revecied_at_unix_to_iso.format": "yyyy-MM-dd'T'HH:mm:ssZ",
"transforms.revecied_at_unix_to_iso.target.type": "string"

How to make SMT work with mongo-kafka connect?

wan · 2020-02-18T05:09:04.416Z

Hi @Sendoh_Daten,

The messages from mongo-kafka source connector are Extended JSON strings. What does the document look like in the collection ? Especially for fullDocument.received field.

Looking at Kafka Connect TimestampConverter SMT, the format property can used to generate the output or parse the input. Could you clarify whether you’re trying to generate a string date in the format of yyyy-MM-dd'T'HH:mm:ssZ ?

Regards,
Wan.

Robert_Walters · 2020-09-09T21:18:03.794Z

The next version of the connector will support formats other than string so you will be able to do SMTs

RUBEN_RENEDO_AGULLO · 2024-06-25T10:26:54.308Z

Sorry, but I have the same error like [Sendoh_Daten] .
The connector show this error: Caused by: org.apache.kafka.connect.errors.DataException: Only Struct objects supported for [converting timestamp formats], found: java.lang.String.

With this configuration:
transforms: Timestamp
transforms.Timestamp.format: ‘yyyy-MM-dd’‘T’‘HH:mm:ss.SSSXXX’
transforms.Timestamp.target.type: Timestamp
transforms.Timestamp.type: org.apache.kafka.connect.transforms.TimestampConverter$Value
transforms.Timestamp.field: eventDate

With what version or when is this problem expected to be corrected?