3
Jan 2023
2 / 8
Jan 2023
7d ago
Meinrad_Hermanek
Jan 2023

Hi

We have to import several different JSON Exports into MonogDB using Go. What we do is fetching the JSON from a database, unmarshalling it into a struct and save the struct into a Mongo collection. This works. The problem I struggle is the handling of the dates. The dates are stored in the format “YYYY-MM-DDTHH:MM:SS” in the incoming JSON. When using the normal unmarshal function it claims with a parsing time error “parsing time \"\\\"2023-01-10T08:56:02\\\"\" as \"\\\"2006-01-02T15:04:05Z07:00\\\"\": cannot parse \"\\\"\" as \"Z07:00\"”.

For this reason, I created then an own type

type MyDate time.time

and a implmented the UnmarshalJSON interface.

func (v *MyDate) UnmarshalJSON(b []byte) error {
   parsedTime, err := time.Parse("\"2006-01-02T15:04:05\"", string(b))
   if err != nil {
      return err
   }
   *v = MyDate(parsedTime)
   return nil
}

So far so good, I got my date in my Go struct

type MyData struct {
   FieldS string
   FieldI int64
   FieldD *MyDate
}

When I now try to save this struct to MongoDB, it creates an empty Object for MyDate. Therefore I implmented the MarshalBSON interface:

func (v FcsDate) MarshalBSON() ([]byte, error) {
   return bson.Marshal(map[string]map[string]interface{}{
      "$date": {"$numberLong": fmt.Sprintf("%v", time.Time(v).UnixMilli())},
   })
}

This works, and my date is stored in the collection, but as it seems, it’s not really a date. When checking with Compass, it’s shown as an object, when checking with Atlas it’s shown as a date. When trying to project it using an aggregation pipeline it claims with “can’t convert from BSON type object to Date”. When I look into another collection from another running application holding a date, then the JSON representation is the same {$date:{$numberLong: nnn}}

What am I doing wrong? I could use string instead of a date, and then everything works technically fine, but my date is still not a date, it’s a string. What solution would you suggest to solve this issue?

3
Matt_DaleMongoDB Employee
Jan 2023

@Meinrad_Hermanek thanks for posting and welcome!

The implementation you posted has two issues:

  1. The string that looks like {"$date": ... } is actually the Extended JSON representation for a BSON “UTC datetime” field, not the BSON representation. By default, a Go time.Time is marshaled to a BSON “UTC datetime” field, but the current MarshalBSON function is actually returning a nested document with a single field called $date.
  2. Implementations of the bson.Marshaler interface (i.e. the MarshalBSON function) must return an entire BSON document. However, what you want to do is override the encoding for a field, not create a nested document. To encode an individual field in a BSON document, you actually want to implement the bson.ValueMarshaler interface instead.

To resolve those two issues, replace MarshalBSON with MarshalBSONValue and return the default BSON field encoding for a Go time.Time value:

func (v FcsDate) MarshalBSONValue() (bsontype.Type, []byte, error) {
	return bson.MarshalValue(time.Time(v))
}

See an example on the Go Playground here.

15 days later
Meinrad_Hermanek
Jan 2023

@Matt_Dale thanks again for your hint.

Now, when I try to Unmarshal this bson value back into my FcsDate, I’m struggling again. I tried following:

func (v *FcsDate) UnmarshalBSONValue(t bsontype.Type, b []byte) (err error) {
	log.Printf("UnmarshalBSONValue: %v, %v", t, b)
	ts := int64(binary.BigEndian.Uint64(b))
	// ts is in nanoseconds, so we convert it to seconds
	*v = FcsDate(time.Unix(ts/1000/1000/1000, (ts%1000)*1000000).UTC())
	log.Printf("FcsSate is: %v", *v)
	return nil
}

This returns a date, but it’s wrong. For example it returns “1970-06-06T09:59:31.880Z” instead of “1997-05-23T00:00:00.000+00:00”.

Do you have any advise?

Regards

Meinrad

Matt_DaleMongoDB Employee
Jan 2023

Hey @Meinrad_Hermanek thanks for the follow-up question! To unmarshal the BSON value, create a bson.RawValue and use that to unmarshal the bytes into a Go time.Time value:

func (v *MyDate) UnmarshalBSONValue(t bsontype.Type, b []byte) error {
	rv := bson.RawValue{
		Type:  t,
		Value: b,
	}

	var res time.Time
	if err := rv.Unmarshal(&res); err != nil {
		return err
	}
	*v = MyDate(res)

	return nil
}

See an example on the Go Playground here.

P.S. I’m surprised that there isn’t a corresponding UnmarshalValue function in the bson package. That seems like an oversight and definitely makes satisfying the ValueUnmarshaler interface a lot less intuitive. There is an open Jira ticket for adding an UnmarshalValue function (see GODRIVER-1892) that I will suggest the Go driver team prioritize (I work on the Go driver team).

2 years later
15 days later
Matt_DaleMongoDB Employee
7d

@Shankar_Nagarajan Here’s the above example adapted for Go Driver v2:

type MyDate time.Time

type MyData struct {
	FieldS string
	FieldI int64
	FieldD *MyDate
}

func (v MyDate) MarshalBSONValue() (byte, []byte, error) {
	typ, data, err := bson.MarshalValue(time.Time(v))
	return byte(typ), data, err
}

func (v *MyDate) UnmarshalBSONValue(typ byte, data []byte) error {
	var res time.Time
	if err := bson.UnmarshalValue(bson.Type(typ), data, &res); err != nil {
		return err
	}
	*v = MyDate(res)

	return nil
}

See the full example in the Go Playground here.

Note that the signature of UnmarshalBSONValue and MarshalBSONValue in Go Driver v2 use byte instead of bsontype.Type, but are otherwise the same.


Want to read more? Browse other topics in Drivers or view latest topics.