Get answers, ideas, and support from the Apigee Community

Schedule a thread in apigee edge

Hi, I have a requirement to schedule a thread/job to connect to the backend server for every minute just to verify whether it is up or not. If it is down then wanted to retry to connect to the server for every 10seconds. How can we achieve thsi requirement in apigee edge proxy? By using node-cron js? Thanks!

3 Answers

Today, the nodejs code that you embed into an Apigee Edge proxy starts running as soon as you deploy your api proxy.

Within that code, you will usually place a call to http.listen() , or something similar.

Before or around that, you can simply make a call to setTimeout(callback, delay). This allows you to specify a function that will be invoked after a specified time period. That function can do whatever it wants to do, including sending out HTTP requests.

OK, so the setTimeout()/setInterval() trick works great, and the background job starts up as expected and runs without any obvious timeouts.

But when I look at the logs, there's *two* servers working on our backend, and the background job starts up on both at once. What I really want is a way to schedule a job to run on exactly one backend server at a time. I could do this by adding in an HTTP handler and triggering that from one of the many third-party cron services, but I'd rather avoid third parties if possible.

Is there any way to do this from within Apigee? It would seem like an easy thing to add.

Yep, there are multiple servers running your nodejs code, for redundancy. which means multiple servers running your setTimeout() / setInterval. But, while you see 2 servers today, there is no guarantee that there will be two tomorrow, or 3 or 1 or 7. Apigee doesn't document the number of servers as part of the terms of service.

If you want to select a *Single* node, of N>=1, then you need to have an election of of those nodes. I think the raft protocol would suit. There is an Apigee Cache whiich can support inter-process communication, but I have not yet implemented a raft election among the nodes.

If you decide to do this, please post the code , possibly on github.

Remember that this is sort of outside the scope of the stated purpose of Apigee Edge - which is to be an API Proxy.

I considered doing something with locking so that whichever process grabbed the lock first would 'win', but it seems like a weird thing to implement on top of Cassandra (unless you're running with consistency=ALL) The cache sounds like a better possibility but again, not really what it is meant for ...

Primarily what we're doing is an API Proxy, but we're using it over the BaaS as well so I was hoping it could be used to update the BaaS. Sadly, the data source we're using isn't conducive to a proxy-and-cache strategy, so we're stuck doing load-and-query.

Scheduling the update from a schedule is a pretty easy option ... we don't need full consistency, just to have the update run from a single location, so the usual load balancer behaviour on a POST should be fine. I've set this up and it works, but it'd be great if there was a way to do it from within Apigee rather than a third party service.

I am using node-schedule and calling one of my apigee edge proxy at every 24 hours and the scheduler proxy I am calling once from postman. But sometimes it get stopped, not sure about the reason, may be due to apigee maintenance or message processor is down. How can I make sure that once a proxy is called in scheduler and it will never get stopped until I forcefully undeploy it?

Yes you are right about node-scheduler. I have used to schedule a proxy to run at 2PM IST daily and its working.

I have a new requirement and that is, as I have scheduled a proxy to run at 2PM IST and that proxy is reading the csv file from S3 bucket and converting csv to json and finally loading the json to API BaaS. But now I need to make sure that the proxy loading has to be completing in 30minutes. If it is not able to complete the loading in 30minutes then I need to KILL the job forcefully.