Background API Tasks

Is there not a good way to put some of the processing for API's like create-system in the background? It seems like it does everything upfront which causes an API call to take a very long time, especially if for instance an image needs to be copied over.

This results in WHMCS handing for a very long time when "Accept Order" is pressed due to it waiting on an API response.

I just think some of this should be looked at. Solusvm from experience seems to use some closed-source c daemon program that handles these tasks and uses an admin log on the panel if there are setup errors.

Thanks.

Status: 
Closed (fixed)

Comments

I suppose we could have asynchronous APIs that have to be polled for completion, but that would be harder for the average caller to use. On the other handle, WHMCS could have made the call asynchronous on it's size so that the provisioning UI doesn't stall.

tpnsolutions's picture
Submitted by tpnsolutions on Sun, 02/23/2014 - 21:22

Jamie,

Just a thought on the topic...

Why not add a "PayPal IPN-like" method where the call is sent to the background and a "return URL" is specified where Virtualmin would send the response to when complete...

Perhaps to keep things flexible have a flag which determines whether the API is called in the foreground for sends the request to a background task.

-Peter

Not sure how WHMCS would do that. I had to write a custom provision module and use php file_get_contents with a custom timeout setting for API calls as CURL was giving me fits.

Issue there is whmcs needs to know if it was successful or not, but thats only known after a call is completed via the json status key. create-system for example can hang for 10 minutes depending on the actions that are needed to be taken.

tpnsolutions's picture
Submitted by tpnsolutions on Sun, 02/23/2014 - 23:20

pcfreak30,

Sorry, I don't use WHMCS as I have built my own custom scripts for interacting with the API. Sorry if my suggestion won't work for needs, it was just something I thought up at the time. Cheers!

-Peter

My comments were not towards you tpnsolutions, but to JamieCameron. Seems you can't nest comments in the tracker.

tpnsolutions's picture
Submitted by tpnsolutions on Sun, 02/23/2014 - 23:26

Hmm, is this nested???

*** apparently not ***

-Peter

I just used a stopwatch to count how long an API call took. It was around 7 minutes that WHMCS hanged on creation. If a timeout occurs somehow it will not be good for automation from a payment callback or such.

I think this could be implemented without too much work in Cloudmin .. I'll look into it.

That said, WHMCS would have to be changed to actually make use of the async API, and change its UI to report success or failure in an appropriate way.

WHMCS is closed source, so a queue could be implemented to work around its limitations with some sort of polling. For now though I can't really proceed on integration until I can get a reliable response time on the API. In some instances I have oddly gotten hangs resulting in php memory maxing.

So here is how I see SolusVM working. When the create api command is ran, it does a few preliminary checks. Thinks like IP space, node space, if the node is up that would be direct errors.

From there it hands the processing off to a background program and returns successful. Instead of hanging the api for the whole creation, its backgrounded and issues go to a system log.

the api call doesn't take more then 10 seconds, while cloudmins method can hang for 7-10 min.

That would be another option - the API call could return success as soon as basic checks are done, but complete the process in the background.

That would be best. I think it should be a switch to enable that behavior since the same script would be ran on CLI. What you can do is just append the switch option in remote.cgi for any api call and refactor commands to accept it or just ignore it.

Ok, the next Cloudmin release will support a background flag that can be used to put long-running operations like creating and cloning a system into the background (once initial checks are complete).

Would it be possible to get a patch for that when implemented?

Sure .. but you'd also need changes in WHMCS to use it, assuming that's where you are calling the API from.

I know. I was writing a custom module anyways.

Ok - so other than VM creation, are there any other long-running tasks you'd like the ability to background? The API also supports this for cloning and moving VMs.

WHMCS does create, terminate, change package/upgrade/downgrade (we enter manual settings for each product and dont use CM plans), change password, start up, shutdown, reboot.

For all of those it waits on the API. So if any of them have a chance at running for a long period they should support being backgrounded.

Also have you implemented the api by IP yet as I assume the changes will modify the api .pl files too of which we have had to modify ourselves for that. Trying to present a clash of modifications.

Ok, most of those are pretty quick - assuming the VM startup hasn't hung, they should take less than a minute. Only operations that involve a lot of disk IO are slow.

I have implemented API lookup by IP already, so I can send you a fix that includes that feature. Remind me which Linux distro your Cloudmin master is on?

Ok, I have emailed you an updated Cloudmin RPM package.

Also does the rpm include the past changes I have brought up? Is it a copy off git or a mix of things?

It includes all the changes for tickets that I have marked as fixed up to this point.

Could you add the background to the delete-system command? For some reason the call always hangs over remote API even though it completes.

Thanks.

Deletion should be pretty quick already - how long does it take if you delete a VM from the command line?

It doesn't take long, its just the web call hangs indefinitely and I have no idea why. Thats the only command that consistency acts like that.

Does the VM being deleted have a system owner account associated with it?

Yes it does but it is the only VM associated to the user and the user is deleted in a second API call if no other VM's exist with it.

The only way I could see the API command hanging is if Webmin gets restarted as part of the VM deletion process (which you should be able to see in /var/webmin/miniserv.error).

Automatically closed -- issue fixed for 2 weeks with no activity.