If I update my data federation storage configuration, when will other connections see that change?

Devin_Hilly · 2023-12-14T21:29:39.275Z

If I’m connected to my data federation instance, and there are other connections open or currently running queries, when will those other connections see the new storage configuration? I ran some tests with the Go Driver where I changed the path for a collection’s S3 data source. The client that made the update sees the new data immediately. The second client only sees the data from the new path after 5 minutes, but I’m not sure if this is happening on the backend, or if the Go Driver is closing and re-opening the connection after 5 minutes. It seems to me that disconnecting and reconnecting usually refreshes the view of the data, but not always - if I recall correctly, data federation caches the storage configuration across connections for a small period of time, but I can’t find this in the docs. It also seems like the second client will see the new data if it runs storageGetConfig after the update has been made but before running a query, but this doesn’t work for my use-case unfortunately.

Irwin_Dolobowsky · 2024-01-26T18:24:29.730Z

It can take up to 5 minutes for the changes to propagate across the system.

Thanks!
Irwin