AnnouncementIntroducing MongoDB 8.0, the fastest MongoDB ever! Read more >>Introducing MongoDB 8.0, the fastest MongoDB ever! >>

Keller Williams Transforms Into a Technology Innovator with MongoDB

INDUSTRIES

Real Estate Services
eCommerce & Retail
Consumer Services

PRODUCTS

MongoDB Atlas
Atlas Search
Atlas App Services
MongoDB Charts
MongoDB Compass

INITIATIVES/USE CASES

Business Agility
Data as a Service
Analytics
Catalog
Mobile

CUSTOMER SINCE

2018
INTRODUCTION

A global innovator harnessing the power of technology to build relationships

Keller Williams is the world‘s leading real estate company.

Or, at least, that is how it used to be known.

Co-Founder and now Executive Chairman Gary Keller said in 2018 that the company wanted to be recognized as a technology company. That might seem counterintuitive for a company founded in a single office in 1983 on the core belief that real estate is all about relationships. In truth, however, Keller Williams is masterfully synergizing the two, using technology to deliver a seamless user experience so agents are free to focus on what the founders knew matters most: fostering meaningful relationships.

Today, Keller Williams is a technology innovator that in 2020 closed on 1.2 million homes for $407.4 billion in sales volume and earned $10.5 billion. The company has more than 190,000 associates around the globe. The Keller Cloud platform consists of two main components: Command and Consumer. A smart CRM-plus solution, Command is a unified platform that supports internal teams with everything from lead generation to database management. Command also contains a consumer-facing component that streamlines home buying and selling processes, enabling agents and customers to interact and manage listings, contacts, marketing profiles, campaigns, and more.

Jim McClarty, Software Architect, Keller Williams, is on the Consumer team, serving up the real-time data that is the driving force supporting all KW.com searches; 182,000+ personalized sites (one for every agent); and much more.

The technology focus at Keller Williams (KW) is now part of the company’s DNA, said McClarty. “We build technology. That means we hire technologists.”

Technology is also key to what McClarty said is another core precept at KW: helping everyone take their performance and careers to the next level. This core belief is reflected all the way to the executive suite. In February 2021 KW named an executive who was previously its Co-Director of Growth as its new leader: Marc King, President, Keller Williams.

Co-Founder and now Executive Chairman Gary Keller

Co-Founder and now Executive Chairman Gary Keller wanted KW to be recognized as a technology company.
MongoDB is helping make that a reality.

THE CHALLENGE

Making the right data instantly available to agents and customers

KW manages more than 500GB of listing data totaling approximately 70 million records across two databases. One database, Fast Facets, supports faceted search so consumers and agents can quickly obtain information about many properties. The other database, Master Dataset, or MDS, contains all of the property details that come into play once someone is interested in a particular property and wants to drill down to learn more.

KW ingests data from real estate multiple listing services (MLS) across the US. It was running massive and complex queries that ran slower than what was optimal to support a real-time user experience for either agents or customers. That was due in part to the reality that even relatively simple queries, such as map results showing all properties within a given area, had to access MDS.

KW needed to make changes in how its Command and Consumer applications were accessing and leveraging data. The company needed to evolve its data infrastructure such that connection strings would start using the smaller, faster database (Fast Facets) and stop using the larger, slower database (MDS).

THE SOLUTION

A microservices architecture built on MongoDB

When KW started on the path from real estate agency to technology titan, it had already been running its own self-managed instance of MongoDB. “Part of the reason I was excited to join Keller Williams was my years of experience using MongoDB with a startup in Austin [TX, USA], and I wanted to get back to it,” said McClarty.

The team decided rather than continuing to manage its own MongoDB instance, it wanted to focus time, attention, and resources on adding value across the organization.

Shifat Adnan, Co-founder, Chief Technology Officer at Pathao

Jim McClarty, Software Architect, Consumer Team, Keller Williams

So KW mapped out a new data strategy on MongoDB Atlas. It launched its Consumer application on MongoDB Atlas in late 2018/early 2019 and migrated Command from an alternative solution to MongoDB Atlas soon thereafter. “We executed a replatform to MongoDB Atlas,” explained McClarty. “We are now running four microservices on Atlas, and everything is working so much better now.”
“We executed a replatform to MongoDB Atlas. We are now running four microservices on Atlas, and everything is working so much better now.”

Jim McClarty, Software Architect, Consumer Team, Keller Williams

“We made the decision to give our developers the best way to work with data — freeing developers to develop — by outsourcing the management of the data platform,” said McClarty. “Our strategy was to democratize access to the data infrastructure while freeing ourselves from the burden of data management by letting the experts at MongoDB manage the data clusters and machines for us.”
“We made the decision to give our developers the best way to work with data — freeing developers to develop — by letting the experts at MongoDB manage the data clusters and machines for us.”

Jim McClarty, Software Architect, Consumer Team, Keller Williams

KW uses Atlas App Services Triggers to maintain active listings in a MongoDB Collection. “When a record comes in, we’re triggered on the listing and it flows into the smaller database (Fast Facets),” explained McClarty. An array of key value pairs for all facets enables quick lookups.

Using the Trigger, the team sets up a MongoDB Materialized View to proactively shape its database as it wants instead of simply reacting by using MDS data as it comes in. This enables the team to filter the data down quickly and easily, subscribing to the services it wants and aggregating the data. “With MongoDB Materialized Views, I am no longer beholden to the decisions of developers who went before me. If I need data in a different shape, I can get it done quickly and easily.”

“With MongoDB Materialized Views, I am no longer beholden to the decisions of developers who went before me. If I need data in a different shape, I can get it done quickly and easily.”

Jim McClarty, Software Architect, Consumer Team, Keller Williams

McClarty said Triggers replace the functionality that would normally be covered in a cron job and a lot more because they are also event-based, and because you can have Triggers processing functions and even writing out data as a result of an aggregation you’ve already worked with. “Aggregation gets the data into just the shape your application needs. Now you can build out your data into a view that’s materialized, and have indexes built off the data. That has been tremendous.”

“I don’t know how we would have done this without Triggers,” he added. “We get listings from all connected sites and systems every 15 minutes and now, instead of experiencing data loss, we are equipping agents and consumers with the most complete and accurate information about properties in real time.”

“I don’t know how we would have done this without Triggers. Instead of experiencing data loss, we are equipping agents and consumers with the most complete and accurate information about properties in real time.”

Jim McClarty, Software Architect, Consumer Team, Keller Williams

McClarty characterized MongoDB Charts, the native visualization tool for MongoDB, as another standout feature. “We use MongoDB Charts a lot,” he said. “MongoDB Charts is helping us manage everything from core functions — for example, whether we’re getting data ingestion consistently — to more user-facing considerations, such as how many consumers currently using the system are associated with a given agent.”
“MongoDB Charts is helping us manage everything from core functions — for example, whether we’re getting data ingestion consistently — to more user-facing considerations, such as how many consumers currently using the system are associated with a given agent.”

Jim McClarty, Software Architect, Consumer Team, Keller Williams

“MongoDB Charts helps me give our product managers the information they are always asking for, and best of all, I can create a chart once and they can self-serve going forward,” said McClarty. “Thanks to Charts I don’t have to run queries for my product managers to report back to upper management. I can configure the charts once and get on with my other work.”

“MongoDB Charts helps our teams visualize what is happening in the data,” he added. “So, for example, when the pandemic hit, we could assess: What steps did you take and what were the results? Did it increase our reach or surpass where we were before?”

“MongoDB Charts helps me give our product managers the information they are always asking for, and best of all, I can create a chart once and they can self-serve going forward.”

Jim McClarty, Software Architect, Consumer Team, Keller Williams

The Aggregation Pipeline Builder in the native GUI for MongoDB, MongoDB Compass, enables McClarty and his team to quickly build aggregation pipelines to process data. Documents in a MongoDB Collection or View pass through stages where Compass processes them into a set of aggregated results, and the team can change the stages and results to suit its needs. “When I was first using MongoDB years ago I wrote a lot of FIND queries. Then as I got more advanced, I started to use the Aggregation Pipeline Builder in MongoDB Compass. Now I point our people directly to the Aggregation Pipeline Builder and don’t even bother with FIND queries,” said McClarty. “The Aggregation Pipeline Builder is a game-changer.”

Other capabilities built into Atlas and across the platform are covering a range of technological bases for KW so the team can stay focused on adding value. In McClarty’s words, “The world's best chefs may know how to build a fire, but they don’t usually do it when they are preparing a 3-Star meal.”

McClarty said, “The autoscaling, alerts, and backups are all things I might be able to put on an Ops hat and do, but having someone else do that means I can focus on the needs of the customers.” He added that the geolocation and integrated search capabilities built into Atlas address various needs, including the ability to share with someone how many open houses are happening right now, or how many new users there are in a given state.

The robust features and flexibility of Atlas App Services are pointing to new ways to use the product, said McClarty. “We recently used Atlas Functions to avoid a full rollback waiting for a new rule to be ready. We simply put in the business logic to make it happen — it took literally two minutes to prepare — and our live data kept performing flawlessly straight through.”

“We recently used Atlas Functions to avoid a full rollback waiting for a new rule to be ready. We simply put in the business logic to make it happen — it took literally two minutes to prepare — and our live data kept performing flawlessly straight through.”

Jim McClarty, Software Architect, Consumer Team, Keller Williams

Other features that are saving time and enhancing workflows include MongoDB’s alerting when an index might help a certain type of query, and the GUI for user permissions control.

In KW’s view, the benefits transcend products and features.

“The documentation with MongoDB is great; I can always find what I need. For many other products I will typically go to other resources first, but for MongoDB I come to your documentation first.”

KW also finds high value in MongoDB educational offerings. “I definitely recommend MongoDB University,” McClarty explained. “We have all of our developers take
MongoDB Basics 001 and more advanced courses. Even our QA team and other colleagues including our Scrum Master have taken MongoDB Basics 001.”

“For us, it all adds up to the fact that MongoDB is a true developer data platform (Atlas),” said McClarty.

“MongoDB is a true developer data platform.”

Jim McClarty, Software Architect, Consumer Team, Keller Williams

THE RESULTS

Incredible transformation with MongoDB resulting in market-leading growth

KW has worked with MongoDB to achieve a remarkable transformation from real estate company to technology innovator. Empowering agents, buyers, and sellers with the right mix of real-time data that is easy to navigate helps all parties be agile, better-informed, and able to move faster, creating millions of happy homeowners each year.

KW has transformed itself into a technology innovator creating millions of happy homeowners each year.

This transformation has also helped fuel substantial growth outpacing the real estate market as a whole. In 1Q2021 KW transactions were up 21.3% YoY versus the industry average of 13.9%, and sales volume increased by 39.7% as compared with the industry average of 27.6%. KW contracts written volume was up 35.1% YoY. That growth is likely to accelerate as KW expands its worldwide presence beyond North America. Consumers and agents will soon be able to search for properties in other world regions on KW.com, supported by the underlying developer data platform (Atlas) with integrated Atlas Search built on top of Apache Lucene.

One of the more notable potential future developments KW is currently considering is using Atlas App Services to serve up that new worldwide listing service, which would reduce complexity by eliminating an entire microservice. This is made possible because Atlas App Services offers the ability to manage GraphQL on the MongoDB side. Atlas App Services may also provide the backend in KW’s MEAN Stack as a replacement for Express.js.

The challenge in making these moves is the cost/benefit analysis with regard to translating business rules that must occur to make it all work. The plus side, per McClarty: “Adding some of these things could have us living happily in MongoWorld someday without having to deal with much of anything else.”

As mentioned earlier in this story, part of the attraction for McClarty to join Keller Williams was his years of experience using MongoDB at another company. When we spoke with him for this story, McClarty extended a similar thought to potential job-seekers: “If anyone wants to learn more about MongoDB and other cool technologies, Keller Williams is a great place to do that.”

“Adding some of these things could have us living happily in MongoWorld someday without having to deal with much of anything else.”

Jim McClarty, Software Architect, Consumer Team, Keller Williams

What will your story be?

MongoDB will help you find the best solution.