Extremely slow execution of an external dependency function

Sumedha_Mehta1 · 2021-07-09T15:43:44.292Z

Hi all - we’re still investigating why the stripe dependency is slower, unfortunately it’s not as straightforward to fix as hoped. I do want to call out that our dependency functionality is still in beta and if you’re having issues with them, there are immediate work arounds (such as calling the API directly). However, we’re carefully monitoring any dependency bugs that show up and filing tickets to address them. If any other part of Realm is not working as expected, please let us know!

Byron_Sorrells · 2021-07-09T15:52:38.759Z

Thanks for the quick updates @Sumedha_Mehta1. Everything else is working great. And calling the API directly works fine for now.

Harry_Heffernan · 2022-02-19T01:43:31.358Z

Is there any update from the MongoDB team on the usability of the Stripe SDK? I am also seeing timeouts when I try to do anything.

andrefelipe · 2022-02-22T21:50:10.369Z

This is not Stripe only, I have the same issue with Algolia.

In my case it just times out and don’t update Algolia.

My function is a simple database trigger, that updates the object to Algolia with “saveObject”. The function runs fine, nothing is slow regarding the required dependency, then, at the index.saveObject the functions just hangs out forever, and Algolia is not even updated.

My guess it’s something regarding network. But looks very much like this issue with Stripe.

It’s a very important fix, basically kills the Realm Functions, and I am about to integrate Stripe too.

andrefelipe · 2022-03-05T20:47:54.984Z

Hello MongoDB, did you get improvements on this executions times? I am about to integrate payments through Stripe into my app, and I can’t loose sales. If I loose sync to Algolia, I can recover later, but that is not feasible for sales.

I use Realm Password Auth for my app, so relying on Realm Functions is wanted, where I can check the auth state. So having Stripe into the Realm Function is very very needed.

Thanks, I want to rely as much as possible into the MongoDb platform.

Rishi_uttam · 2022-03-06T04:42:12.341Z

Realm Functions / Mongo Triggers have had little or no release updates (or atleast not disclosed) it seems this is not a focus they want to target as a lot of q’s remain unanswered. I myself have moved on to Lamda and Cfloudflare functions to solve my problems… Yes its a pain as Realm Auth was good, but that too has seen no improvement and no mentioned of passwordless auth.

If you are a enterprise customer you should get a reply or try online chat support.

andrefelipe · 2022-03-06T13:57:40.138Z

Thanks @Rishi_uttam for you time to reply.

Indeed very sad for MongoDB.

Rishi, may I ask, on your move to Lambda and Cloudflare, do you still use the Realm Auth? Did that work?

I believe I can confirm the access token is valid with this API below, just would like to know if you used it and if that worked fine? https://docs.mongodb.com/realm/reference/authenticate-http-client-requests/#verify-a-client-access-token

I have 4k users on Realm Auth, so really can’t move to another provider too soon.

Thank you very much.

Sahmed_Matsko · 2022-03-07T09:02:17.684Z

I have the same issue using the AWS-SDK in MongoDB functions. When I try to upload an image to a S3 bucket it takes around 30 to 60 seconds! When I try this on my local machine or a different location it takes around 150ms.

Rishi_uttam · 2022-03-07T09:15:31.851Z

uploading a object to s3 shouldnt be slower as you are uploading to an s3 url.

Sahmed_Matsko · 2022-03-07T10:31:18.894Z

It is though. This is the code:

let buf = Buffer.from(
         base64.replace(/^data:image\/\w+;base64,/, ""),
         "base64"
       );


const uploadParams = {
         Bucket: "BUCKETNAME",
         Body: BUFFERFILE,
         Key: fileName,
         ContentEncoding: "base64",
         ContentType: "image/jpeg",
       };

       s3.upload(uploadParams, function (err, data) {
         if (err) {
           console.log(err);
           console.log("Error uploading data: ", data);
         } else {
           console.log("succesfully uploaded the image!");
         }
       });

andrefelipe · 2022-03-07T12:21:54.153Z

Maybe this all looks like networking issue. In fact, Stripe or Algolia clients would, at the end, just call an URL too.

I had problem with upload too, but it might have been related to input timeout, which was 30s and was raised to 60s as per this thread: Realm functions 30s input timeout

Now back to us, I also tested a Ream App deployed Global vs a Local, just in case. And no difference. The Algolia saveObject just reaches “execution time limit exceeded” in both apps.

Looks like I should give up Realm Functions.

Rishi_uttam · 2022-03-07T12:22:38.204Z

Ahh you’re not uploading from a browser, sorry my bad. your uploading from node. I wouldn’t do that in a serverless function, as wouldn’t you be paying for a long running task this way? why not upload from the client directly.

Sahmed_Matsko · 2022-03-07T12:48:10.808Z

I am afraid my API keys will be exposed. If I upload it directly from the client how can I make sure the API Keys are not exposed??

Rishi_uttam · 2022-03-07T13:09:22.949Z

no no… if you need to upload from the browser uses temp keys that are generated for that user only. i think aws has something called a STS KEY, you first define which path and what the keys can do dynamically, then pass the keys to the browser, and allow the browser to upload… the keys live for a short time say 60 mins.

docs.aws.amazon.com

Using temporary credentials with AWS resources - AWS Identity and Access...

Use temporary security credentials from IAM STS to make programmatic requests for AWS resources with the AWS SDKs or API calls.

henna.s · 2022-03-10T16:10:40.922Z

G’day Folks, @Rishi_uttam , @andrefelipe @Sahmed_Matsko,

I acknowledge the frustration with the issues with dependencies in Realm functions.

The Engineering team is working on these issues and please expect significant improvements in the upcoming weeks. For your questions and comments on specific packages, we would request to open a new topic to avoid confusion.

For updates on AWS S3, please follow this thread:

AWS services moving forward - 3rd party services depreciation Atlas Triggers & Functions

Thanks for the feedback! We are continuing to work on more performance improvements and I will update this thread as changes are released.

I will update here for the release updates on Stripe.

Appreciate all your patience with us.

Cheers,

andrefelipe · 2022-03-11T16:30:42.079Z

Thanks @henna.s those improvements will be a much appreciated!

henna.s · 2022-03-14T22:25:34.340Z

G’Day, Folks, @Rishi_uttam , @andrefelipe , @Sahmed_Matsko ,

I have amazing news… The engineering team have made performance improvements and Stripe, Algolia and Axios run much faster than before.

Could you please try and confirm?

We would appreciate your feedback if you have a different package to test out as well.

Looking forward to your response

Cheers,

andrefelipe · 2022-03-15T00:35:24.609Z

Wow, @henna.s thank you so much for delivering in a timely fashion.

Here are my readings regarding Algolia:

Algolia “update a single object” using raw REST HTTP:

301ms
302ms
300ms
278ms
273ms
296ms
297ms

Algolia “update a single object” using official NodeJS client “algoliasearch”:

299ms
278ms
281ms
299ms
294ms
291ms
277ms
302ms
298ms
299ms

So simply put, you guys nailed it.

Thank you very much for the attention and the solution.

(can’t say about the Stripe yet, but I must tackle that this week, so, also extra happy to rely on MongoDB)

henna.s · 2022-03-24T15:53:35.109Z

A post was split to a new topic: Buffer Performance Improvements

system · 2022-03-29T15:54:28.665Z

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.