Alex Bevilacqua

3 results

Improving the Node.js Driver’s SCRAM-SHA-256 Support

MongoDB always strives to offer best-in-class features, functionality, and security. A number of authentication mechanisms currently exist to verify the identity of a connecting client to your cluster, and when using the Salted Challenge Response Authentication Mechanism (SCRAM) there are two possible hashing functions: SCRAM-SHA-1 and SCRAM-SHA-256 . The MongoDB Driver Authentication Specification outlines that when attempting to authenticate using SCRAM: “If SCRAM-SHA-256 is present in the list of mechanism, then it MUST be used as the default; otherwise, SCRAM-SHA-1 MUST be used as the default [...]”. A MongoDB Server ( mongos or mongod ) can be configured with a list of possible authenticationMechanisms . As a result, MongoDB can be configured to return new authentication mechanisms which can upgrade already running applications to more secure authentication. This is the case when SCRAM-SHA-256 is added to a cluster that previously only supported SCRAM-SHA-1 . Prior to hashing passwords with SHA-256, they will first be prepared using SASLprep . The MongoDB Node.js driver leverages an external library ( saslprep ) for this functionality, which was an optional dependency and only used if available. Though a number of checks were in place to ensure the library was available (and loaded), an edge case was found where these checks could fail and report availability incorrectly. Potential use Most applications won’t experience this issue, however if your Node.js project is being bundled using an alternate bundler (such as webpack ) it’s possible a variation of this issue may surface. If your application was affected, it would be unable to connect to your MongoDB cluster. The stack trace from the error that would be thrown should include a call to continueScramConversation similar to the following examples: { "errorType": "TypeError", "errorMessage": "saslprep is not a function", "stack": [ "TypeError: saslprep is not a function", " at continueScramConversation ([...]/index.js:xxx:yyy)", [...] ] } TypeError: l is not a function at continueScramConversation (/app/webpack:[...]/mongodb/lib/core/auth/scram.js:xxx:yy) Note that Mongoose applications can also be affected, as Mongoose wraps the Node.js driver: TypeError: (0 , o.saslprep) is not a function at continueScramConversation (/app/webpack:[...]/mongoose/node_modules/mongodb/lib/cmap/auth/scram.js:xxx:yy) Next steps The underlying issue was addressed in versions 5.7.0 , 4.17.0 and 3.7.4 of the MongoDB Node.js driver, so depending on the version of the driver being used by your application a minor version update will address this. Upgrading your application’s libraries and deploying to production may not always be possible in a timely fashion. If this is the case and you happen to hit the issue described above a workaround would be to append the authMechanism option to your connection string with a value of SCRAM-SHA-1 as follows: mongodb+srv://xyz.mongodb.net/test?authMechanism=SCRAM-SHA-1 This will force the driver to attempt authorization using the SCRAM-SHA-1 hashing algorithm. Note that connection string changes would still require the application to be restarted for those changes to take effect.

August 23, 2023

Changes to the findOneAnd* APIs in Node.js Driver 6.0.0

Do you use the MongoDB Node.js driver? If so, there’s a good chance you use various find() operations regularly. MongoDB plans to release version 6.0.0 of the Node.js driver in August 2023, and we’ve made some exciting improvements to the findOneAnd* operation. With the new driver release, the modified (or original) document targeted by a findOneAnd* operation will now be returned by default. Current state Up until now, as opposed to returning the requested document, this family of API methods would return a ModifyResult , which would contain the requested document in a value field. This design was due to these APIs leveraging the MongoDB Server’s findOneAndModify command and wrapping the command’s output directly. To demonstrate, let’s adapt the code from the Driver’s documented usage examples to update one document in our movies collection using the findOneAndUpdate API. const database = client.db("sample_mflix"); const movies = database.collection("movies"); // Query for a movie that has the title 'The Room' const query = { title: "The Room" }; const updatedMovie = await movies.findOneAndUpdate(query, { $set: { "imdb.rating": 3.4, "imdb.votes": 25750 } }, { projection: { _id: 0, title: 1, imdb: 1 }, returnDocument: "after" }); console.log(updatedMovie); { lastErrorObject: { n: 1, updatedExisting: true }, value: { title: 'The Room', imdb: { rating: 3.4, votes: 25750, id: 368226 } }, ok: 1, '$clusterTime': { clusterTime: new Timestamp({ t: 1689343889, i: 2 }), signature: { hash: Binary.createFromBase64("3twlRKhDSGIW25WVHZl17EV2ulM=", 0), keyId: new Long("7192273593030410245") } }, operationTime: new Timestamp({ t: 1689343889, i: 2 }) } One of the options we set was a returnDocument of after , which should return the updated document. Though the expectation may be that the function call would return the document directly, as we can see this isn’t the case. While the document you’re looking for can be accessed using updatedMovie.value , that isn’t the most intuitive experience. But changes are on the way! What can we do right now? Starting with the Node.js Driver 5.7.0 release a new FindOneAnd*Options property called includeResultMetadata has been introduced. When this property is set to false (default is true ) the findOneAnd* APIs will return the requested document as expected. const updatedMovie = await movies.findOneAndUpdate(query, { $set: { "imdb.rating": 3.3, "imdb.votes": 25999 } }, { projection: { _id: 0, title: 1, imdb: 1 }, includeResultMetadata: false }); console.dir(updatedMovie); { title: 'The Room', imdb: { rating: 3.3, votes: 25999, id: 368226 } } What about TypeScript? If your application uses TypeScript and the MongoDB Node.js Driver, anywhere a findOneAnd* call is made, if the requested document is required it will be accessed via the value property of the ModifyResult . This occurs when includeResultMetadata is not set or when it is set to true (the current default value). Type hinting will indicate the Schema associated with the collection the operation was executed against. As we would expect, when the includeResultMetadata is changed to false inline validation will indicate there’s an issue as the value property no longer exists on the type associated with the result. Attempting to compile our TypeScript project will also fail. TSError: ⨯ Unable to compile TypeScript: index.ts:31:17 - error TS18047: 'updatedMovie' is possibly 'null'. 31 console.dir(updatedMovie.value); ~~~~~~~~~~~~ index.ts:31:30 - error TS2339: Property 'value' does not exist on type 'WithId<Movie>'. 31 console.dir(updatedMovie.value); Next Steps If you’re using the findOneAnd* family of APIs in your JavaScript or TypeScript project, upgrading the MongoDB Node.js Driver to 5.7.0+ and adding the includeResultMetadata: false option to those API calls will allow you to adapt your application to the new behavior prior to the 6.0.0 release. Once 6.0.0 is released, includeResultMetadata: false will become the default behavior. If your application relies on the previous behavior of these APIs, setting includeResultMetadata: true will allow you to continue to access the ModifyResult directly.

August 8, 2023

Technical Services Engineering at MongoDB: Meet Alex Bevilacqua

I've been a technical services engineer (TSE) at MongoDB for two years now, but I wanted to share what the journey of getting started in this role looked like for me. I'm also going to dive deeper into what a TSE actually does and why it is both a challenging and fulfilling career. First, a Bit About Me I have been writing software since I was a kid, starting with some automation tools for my mom's business. I then moved on to building tools to help me cheat at various games I was playing at the time, and eventually I got more into emulator programming and reverse engineering. I guess you could say I've always loved solving problems programmatically, and I've especially enjoyed identifying opportunities for automation and custom tooling. I have been working in application development and software engineering for nearly two decades. I started off writing desktop applications in QuickBASIC and Turbo Pascal, and then eventually in Visual Basic 6, Visual Basic .net, C++, and C# as well. When it was time to shift focus to web development, I began with HTML/JavaScript/Cascading Style Sheets (as we all do), and then moved to Adobe Flash/ActionScript 3, Flex, Python, Ruby on Rails, and Node.js. This led me down an informal DevOps track, because I was finding a need for optimization in the infrastructure layers to which my applications were deployed. And that led me deeper into Linux internals, system administration, and network operations. While I was gaining these new skill sets, my primary focus was always on application development and delivery. Before coming to MongoDB, I was working as a development lead/system architect, but I found that my focus was always being drawn back to solving performance challenges at the infrastructure level. Why MongoDB? I started working with MongoDB on a number of "hobby" projects around 2012. At the time, I really only had experience with RDBMS's, but due to the unstructured nature of the data I was working with, I decided to give this new technology a whirl. I fell in love with the database almost immediately and have since carried it forward to multiple new employers, as well as to contract opportunities and consulting agreements. The low barrier to entry from a development bootstrapping perspective made it ideal back end for proof-of-concept development through to production deployment. As a result of this increased activity with MongoDB, I found myself doing a lot more investigaiton into performance issues and internals (links are to blog posts about challenges I encountered and resolved). Why Technical Services? This was initially very challenging for me, because I had preconceived notions about what "technical services" actually implied. The first thoughts that popped into my head were "technical support," "client support," "tiered customer support," and so forth. While researching this position, I came across a two-part blog post from 2012 by a MongoDB employee who blogged about his experience as a support engineer. I found his reasons for joining MongoDB, such as the kinds of challenges the job poses on a daily basis and how there is a constant push for self improvement and continuing education, aligned with what I was looking for in a new career. What's a Technical Services Engineer on Paper? To answer this question, let's start off by analyzing the job posting that kicked off this journey for me in the first place. So, they're looking for people who are able to solve problems and communicate clearly. This could be a call center gig after all...oh wait, experts in MongoDB, related database servers, drivers, tools, services...hmm, maybe there's a bit more to this. Architecture, performance, recovery, security -- those are a lot more complex than what you would face in a traditional support role. What really sold me, though, was the "contribute to internal projects" statement, which aligned perfectly with my desire for process improvement through custom tooling. By the time I got to this point in the job posting, I was already sold. MongoDB is either trying to staff its first-tier support with ridiculously over qualified employees, or technical services really isn't what I thought it was. I proceeded to fill out the application, attached my resume and cover letter, and crossed my fingers that MongoDB would reach out to me. What's a Technical Services Engineer in Practice? After working with other TSEs for the past two years and having had an opportunity to handle customer cases on my own, I think I can shed a bit of light on what this role really entails. How is it a Support Role? A TSE interacts with MongoDB's clients via a support queue. This allows incoming "cases" to be prioritized and categorized to allow engineers to quickly identify what form of subject matter expertise may be required (indexing, replication, sharding, performance, networking, or drivers, for example). As a TSE, You're responsible for claiming cases from a queue and providing feedback in a timely fashion that is clear, concise, and technically accurate. The types of problems can vary from "How do I...," to "We are preparing for a major sales event and want to ensure we're properly configure" to "OMG EVERYTHING IS ON FIRE!!!" I've had the privilege of leveraging some of my past experience to assist customers through data recovery exercises, and of using new skills I've learned to help with query optimization, programming challenges, performance troubleshooting, and diagnostic analysis. With experience also comes the opportunity to become more integrated with key clients that require more attention. As a trusted advisor, you act as the liaison between the client and MongoDB Technical Services and are more involved in long-term planning to help customers prepare for major events. How is it an Engineering Role? Here's the juicy part of this job. Although replying to client requests is the "deliverable" for a TSE, how you go about reproducing the clients' issues requires a very deep understanding of MongoDB internals, software engineering, network engineering, infrastructure architecture, and technical troubleshooting. Depending on the type of issue, a reproduction is likely in store. These involve re-creating the environment (locally or in the cloud) to either benchmark or replicate the identified client challenge. There is a vast library of tools available to TSEs for these kinds of tasks, but on some occasions, the right tool for the job may not exist. In these cases, you have an opportunity to write your own scripts or tools to parse logs, measure performance, record telemetry, or verify a hypothesis. Although MongoDB doesn't require TSEs to have any programming experience, for those such as myself who come from product engineering, it's refreshing to know there's still an opportunity to scratch the development itch. How Does it Feel Working Here? MongoDB has set a high bar for TSEs with respect to the level of experience and expertise required to join the ranks. Although I'd already had a multi-decade career in software development and architecture, I definitely felt some imposter syndrome as I got to know and interact with my team. Everyone I've had the pleasure of working with up until now has been welcoming and helpful. I've learned a lot about MongoDB's suite of applications, services, and drivers and continue to learn daily. As a programmer I have access to like-minded developers. As an author I have access to like-minded bloggers and writers. As an individual contributor I have access to a wealth of resources and knowledgeable individuals who are willing and able to help me continue achieving new personal and professional goals. Wrap-Up The TSE role continues to be redefined and refined as new MongoDB products come on board and new challenges present themselves. What will likely remain constant, though, is the need for new engineers with the following characteristics: A passion for continuing technical education A willingness to step outside their comfort zone A interest in software engineering A interest in network operations I encourage you to check out MongoDB's available jobs if what I've described here interests you (I swear HR is not putting me up to this), because we could use more engineers like you in our ranks! Feel free to read my personal blog or shoot me an email at alex@alexbevi.com if you have any questions. Interested in pursuing a career at MongoDB? We have several open roles on our teams across the globe , and would love for you to build your career with us!

October 12, 2020