Validation using latest JSON Schema version?

Andrzej_Taramina · 2020-03-27T21:36:08.492Z

When will MongoDB support validation using the latest JSON Schema standard, which is draft 2019-09.

Currently MongoDB only supports JSON Schema Draft 4, which is pretty long in the tooth.

Thanks!

Stennie_X · 2020-03-27T22:12:29.630Z

Welcome to the community @Andrzej_Taramina!

Andrzej_Taramina:

Currently MongoDB only supports JSON Schema Draft 4, which is pretty long in the tooth.

What specific features are you interested in using from the newer JSON Schema drafts?

Updating isn’t straightforward as there are backward-incompatible changes. For example, see: Changes between draft-04 and draft-06.

There are already some tracking issues for features that may be of interest to upvote and watch:

SERVER-42035: Extend JSON Schema implementation to support conditional application of subschemas
SERVER-42036: Extend JSON Schema implementation to support built-in format formats

Regards,
Stennie

Richard_Braman · 2020-06-07T20:36:00.697Z

Thanks for the pointer to this theead. As the JSON Schema spec moves towards RFC recognition, it is helpful if json databae like mongo supports it. It should be the standard way for doing Jaon data validation. Otherwise devlopers can use validation libraries before Creates and updates. But much better if Mongo handles it. I am looking at the HL7 FHIR schema which conforms to draft 6. The combination of json achema and mongos native json document collections can prove a powerful comination in implementing complex data models like FHIR.

Stennie_X · 2020-06-09T11:16:18.473Z

Hi @Richard_Braman,

Thanks for the practical example of HL7 FHIR schema, which currently appears to target JSON Schema draft-06. Draft-06 is newer than MongoDB’s server implementation of draft-04 but older than the latest 2019-09 JSON Schema draft. As noted in my earlier comment on this topic, there are some backwards-incompatible changes between different versions of JSON Schema. Draft specifications are unfortunately a moving target, but I expect JSON Schema will generally have more consistent implementation versions after an RFC has been proposed & adopted by the IETF.

JSON Schema support required for HL7 FHIR would be a good example to include in a feature suggestion on the MongoDB Feedback Engine so others can upvote & comment on the idea.

In the interim, some workarounds would be to either incorporate a JSON Schema draft-06 validation library in your application tier or transform draft-06 schema into declarations compatible with the current MongoDB server implementation. Even with schema validation support at the database server level, I would recommend validating in your application layer to avoid the extra network round trip and provide more informative validation messages. However, it would be more ideal to share a single definition of JSON Schema across multiple validation layers in your stack.

You may also be interested in Building FHIR Applications with MongoDB Atlas, which describes an open source FHIR server (MIT licensed) based on MongoDB.

Richard_Braman:

it is helpful if json databae like mongo supports it

It is worth noting that MongoDB supports JSON-like data structures, but is not a JSON database. MongoDB’s underlying data format is BSON, a Binary JSON-like data format that represents data with richer data types than those available in JSON. For example, BSON has extended numeric types including int32, int64, double, and Decimal128 whereas JSON only has a generic Number (aka double) type. MongoDB extends JSON Schema with a bsonType keyword to specify data types.

Regards,
Stennie

NeilM · 2021-05-28T11:31:44.963Z

@Stennie_X

Sorry to dig up an old thread, but I wondered what was the news on this?

I am currently working through M036 course, and it has a section on JSON schema, so I was hunting around to see what progress MongoDB had made on since v3.6 and it seems like not a lot.

So, is this something that MongoDB are no longer invested in or is JSON schema no longer a thing?

If MongoDB are concerned with backwards compatibility, then I am sure there are many ways in the tools could be setup to handle v0.4 and newer versions, to allow 0.4 and draft x.x to be used side by side, as people migrate there schema’s on to the later versions.

It is really worth sticking with things, and not becoming Google

Thanks

PS - I know M036 is an old course, it does contain some interesting information. I not sure it should be retired. The course implies that MongoDB went with implementing 0.4 as a draft standard, so something being draft, isn’t a problem

Stennie_X · 2021-05-28T12:59:21.254Z

Hi @NeilM,

There hasn’t been any specific movement towards adding JSON Schema syntax from newer drafts beyond the sort of tracking issues I mentioned earlier, but there also hasn’t been any significant demand. Since development resources are finite, product and engineering teams have to choose which features or improvements to invest in.

For example, a highly requested feature has been for the MongoDB server to return more detailed results when validation fails. This validation feature has been added to the upcoming MongoDB 5.0 release.

Excerpt from the MongoDB 5.0 release candidate documentation:

Starting in MongoDB version 5.0, the validator returns detailed error information when a validation condition isn’t met. The error output is exhaustive - all errors are reported, not just the first one.

If there are specific JSON Schema features you are interested in, please provide more information on the MongoDB Feedback site and share the link to the feedback topic here so others can upvote & comment on the idea.

It is not unusual for working groups to publish drafts that are not widely adopted until years after publication. A general caveat on draft standards is that iterations are more likely to remove or change behaviour until a version is declared stable/recommended.

I would also consider validation at the database level as a last resort in a typical multi-tier application. A web application has opportunity to validate in the presentation tier (eg using HTML5 form controls or JavaScript), in the application tier (business logic), and finally in the data tier. That doesn’t mean you shouldn’t validate at each opportunity, but the breadth of feature support tends to mirror that usage progression.

Regards,
Stennie

NeilM · 2021-05-28T16:13:09.000Z

@Stennie_X

Thanks for the response.

I am pleased that all errors are going to be reported.

Hopefully this will be a solid implementation

I do remember looking at the validator in 3.2 and asking about error handling in 2016, which is probably the last time I looked at the feature. I should have nagged a bit more.

As I said early, I only came across the JSON schema, when going through M036 course

Sorin_GFS · 2021-11-09T00:22:24.418Z

Stennie_X:

If there are specific JSON Schema features you are interested in, please provide more information on the MongoDB Feedback site and share the link to the feedback topic here so others can upvote & comment on the idea.

Hi,

I started using validation and soon I realized that there were shortages. Anyway, I think using the MongoDB built-in validation is tens of times faster then using some validation in application layer, therefore I use the built-in validation everywhere. I even cache the validation schema in memory for production environment, after it has previously been adjusted in development environment.

So, my first problem when I use the validation schema in frontend is the lack of support for unknown keywords which would allow me to store some frontend specific needs (like format) in the database.

Stennie_X:

It is not unusual for working groups to publish drafts that are not widely adopted until years after publication. A general caveat on draft standards is that iterations are more likely to remove or change behaviour until a version is declared stable/recommended.

To solve this problem I think MongoDb should start versioning the validation so we can choose which one to use.

But one thing is certain: the ‘jsonSchema’ rfc has evolved massively with the 2019-09 version, and at the same time the OpenApi 3.1 specifications have been adapted with this version. Some random article about that.

So, to recap, taking into account the fact that personnel resources are limited, my priorities would be the following:

versioning the validation (I think it shouldn’t be hard)
allow unknown keywords (easy to implement, and would solve for the moment the problem of currently unsupported keywords)
Add support for harmless keywords like:
- readOnly and writeOnly RFC
- examples RFC
Add support for `Vocabularies for Semantic Content With “format” RFC

These are just a few of the important points, but I think MongoDb shouldn’t be limited to that and should invest time to make validation usable in the frontend too.

Ronald_Bergmann · 2025-02-19T13:55:44.454Z

Can we please get a recent jsonschema version? draft 4 is ancient. I cannot even define a property to always be true for there’s neither const nor enum. For non-trivial schemas this is such a PITA.