1 (One) Minute Cron Jobs :)

[PauloV]

Hello blestars

 

Long time I post in here eheh

 

My request is simple, and me personally and one client that Im working with, asks for CORE implementation, to be able to set, also, Cron Jobs to run in "Interval", by 1 (one) minute and not by the default minimum 5 minutes, that I think is to hight to check for exemple new Incoming Emails on IMAP/POP Support Manager (Pro also)

 

Here is the solution to implement on CORE, only 2 lines of code change in 2 files

 

Edit: [your-blesta-instalation]/app/controllers/admin_company_automation.php

Find on line: 160

$intervals = array(5=>5, 10=>10, 15=>15, 30=>30, 45=>45);

Change to:

$intervals = array(1=>1, 5=>5, 10=>10, 15=>15, 30=>30, 45=>45);

Edit: [your-blesta-instalation]/app/controllers/cron.php

Find on lines 2087/2088:

// The last run date rounded down to the nearest 5 minute interval
$last_run_date = date("c", floor($this->Date->toTime($last_run->start_date)/(60*5))*(60*5));

Change to:

// The last run date rounded down to the nearest 1 minute interval
$last_run_date = date("c", floor($this->Date->toTime($last_run->start_date)/(60))*(60));

Finally, change your server/host blesta CRON task from something like:

*/5 * * * * /usr/bin/php /home/[path-to-your-blesta-instalation]/index.php cron

Change it to:

* * * * * /usr/bin/php /home/[path-to-your-blesta-instalation]/index.php cron

Thanks in advance

 

Keep it up

 

Regards,

PV

 

[Michael]

Bad bad bad idea, we spoke about this before I believe in this forum and it would cause a server overload if you run a cron every minute because it does everything from orders, to emails, downloading tickets.

[PauloV]

Bad bad bad idea, we spoke about this before I believe in this forum and it would cause a server overload if you run a cron every minute because it does everything from orders, to emails, downloading tickets.

 

 

But if you set Only some tasks to one minute like:

 

Download Tickets

Delivery Invoices

Provision Paid Pending Services

 

(I think are the most important tasks to run quickly)

 

The server will not overload, because the outher tasks are "Pending"  and they will not run every one minute

 

Or at least have a RULE to apply some "time" limits on some Modules/Plugins this way the adverge admin will not be able to set by mistake the one minute task of course this will take more time to implement on core files because it will need changes on several core files, in modules/plugins, and maybe database

[Michael]

Still don't agree with it, you will run into lots of problems but we'll wait for the guys to explain better developer to developer.

Download tickets works straight away when you use piping as it doesn't use the cron job.

[PauloV]

Still don't agree with it, you will run into lots of problems but we'll wait for the guys to explain better developer to developer.

Download tickets works straight away when you use piping as it doesn't use the cron job.

 

 

Yes, PIPE works wonderfull, but wen for exemple you use multi company and/or multiple MX servers you have to use IMAP or POP

 

Yes, I whant to see a response from Cody and/or Tyson (Paul also eheh)

[serge]

the idea is really good PauloV, I agree with you that customers can not wait 5mins to get a service activated when it's paid.

 

In this post I also given how I do make 2 mins cron frequency, with a security to prevent  runing a cron when last one is not yet ended:

 

http://www.blesta.com/forums/index.php?/topic/2990-email-not-being-sent-on-invoice-creation/page-2

 

so with PauloV technique, there is diffrent way, and that fines, but 5 mins wait is too long in some case

[Paul]

We don't recommend doing this, but would still love some feedback from people who do this (Has it caused you any issues with other longer running tasks?). Order activation and POP/IMAP tickets are the primary tasks people ask for more frequent cron runs.

[serge]

I will say it's "bleeding edge" so, I agree it's not for everybody.

 

But untill you are enought skilled to understand how these techniques work and also you know how manage/administer your server, so in only such case, such techniques can be tried, but first on a test server.

[activa]

1+ for activating order and checking new emails .

[activa]

1+ for activating order and checking new emails .

[Blesta Addons]

activating order and sending email is the most wanted thing without delay .

[Fantasma]

big +1 -- i've manually added 1 min crons now - thanks Paulo!

[Cody]

1 minute cron jobs are a bad idea. It's too easy for them to pile up and execute on top of each other.

 

If the point is to pull email from SMTP/IMAP server in near real time, create a script that runs constantly and polls the SMTP/IMAP server every 30 to 60 seconds.

 

For example:

<?php

while (true) {
    $window = 60;
    $start = microtime(true);
    
    // TODO: do stuff

    $runtime = microtime(true) - $start;
    // wait until we can run again...
    sleep(max(0, (int) $window - $runtime));
}

[PauloV]

 

1 minute cron jobs are a bad idea. It's too easy for them to pile up and execute on top of each other.

 

If the point is to pull email from SMTP/IMAP server in near real time, create a script that runs constantly and polls the SMTP/IMAP server every 30 to 60 seconds.

 

For example:

<?php

while (true) {
    $window = 60;
    $start = microtime(true);
    
    // TODO: do stuff

    $runtime = microtime(true) - $start;
    // wait until we can run again...
    sleep(max(0, (int) $window - $runtime));
}

 

 

They only "pile up" if there isnt any verification for exemple put a key on database that tell its still running, if so, the cron will execute returning/executing "nothing", to finish and not run in background, and they will not "pile up"

 

Isnt the way that is working right now? they check the time/date on the cron database and if is not finish the cron will not run?

[serge]

As mentioned above, I use this: http://timkay.com/solo/

 

It's dead simple and it's not posible to have more than one cron job at same time, as cron can not be run untill last cron job ended.

 

Because an open port is directly tied to a running process, chances of inconsistency and having to detect and cleanup orphaned PID files, are zero.

[Cody]

They only "pile up" if there isnt any verification for exemple put a key on database that tell its still running, if so, the cron will execute returning/executing "nothing", to finish and not run in background, and they will not "pile up"

 

Isnt the way that is working right now? they check the time/date on the cron database and if is not finish the cron will not run?

 

Yes that's what Blesta does now. I'm not stating that you can't have 1 minute cron jobs that work. I'm just saying, in practice, it's not really a good idea as the risk of a collision is high.

 

 

Plus, why have 1 minute cron jobs when you have can SMTP polling every 15 seconds?

[serge]

In my case, it's more about get paid service activated quick, my cron is 2 mins frequency by the way.

[activa]

In my case, it's more about get paid service activated quick, my cron is 2 mins frequency by the way.

 

also for me, if we put activation instantly in our website , it should instantly  

 

my request goes for activating order and sending email . is a pain wait 5 minute or less to do it .

[Serendesk]

 

1 minute cron jobs are a bad idea. It's too easy for them to pile up and execute on top of each other.

 

If the point is to pull email from SMTP/IMAP server in near real time, create a script that runs constantly and polls the SMTP/IMAP server every 30 to 60 seconds.

 

For example:

<?php

while (true) {
    $window = 60;
    $start = microtime(true);
    
    // TODO: do stuff

    $runtime = microtime(true) - $start;
    // wait until we can run again...
    sleep(max(0, (int) $window - $runtime));
}

 

 

Process monitors (supervisord, daemontools, launchd etc) are a great way to run a looping script like this in the background on a server, just like a daemon process. They allow start/stop/status/start at boot and auto restart on fail. You can avoid cron completely and they are simple to use. You can even use a shell script rather than php to handle the looping part (execute a php script each loop, e.g. /usr/local/bin/php myCronTasks.php).

 

I use one to run multiple and simultaneous "background services", sort of like a single multi-threaded daemon, so tasks like imap, smtp, notifications are near real time. However, there's a few more tricks needed involving a DB or temp files on disk if you want to run multiple doing the same thing at the same time and avoid overlap. If you only need a single service, then php and a process monitor is all you need.

 

There is never a worry about overlap or locking like cron and things just keep on running. If you handle/catch errors properly so the loop never breaks then the service will run forever (use set_time_limit(0) and gc_collect_cycles()).

 

 

I strongly recommend this approach. Use supervisord if you don't mind installing python or daemontools if you don't want any libraries.

 

 

I haven't tried this with Blesta yet. Cody, can you see any issues in simply executing the Blesta cron command inside a loop?

[Cody]

I haven't tried this with Blesta yet. Cody, can you see any issues in simply executing the Blesta cron command inside a loop?

 

You could create race conditions. Each PHP process is spun off in a new thread, so there's no guarantee one finishes before the next is requested. This is especially problematic when a task performs operations that result in the PHP process blocking for IO, such as processing payments, or receiving email. If you were to do this, you would need to verify that the task ended before starting up the next.

[Serendesk]

You could create race conditions. Each PHP process is spun off in a new thread, so there's no guarantee one finishes before the next is requested. This is especially problematic when a task performs operations that result in the PHP process blocking for IO, such as processing payments, or receiving email. If you were to do this, you would need to verify that the task ended before starting up the next.

 

 

Thanks Cody.

 

 

 

I just took a look at /blesta/app/controllers/cron.php, so rather than execute the cron job in the loop, create an instance of the Cron class and do something like this in your loop:

require('YOUR INSTALL DIR/blesta/app/controllers/cron.php');

//re-use
//$cron = new Cron();

//your loop
while(true)
{
    try
    {
        //or new every time
        $cron = new Cron();
        $cron->index();
    }
    catch(Exception $e)
    {
        //log etc
    }
    //your sleep/interval stuff etc
}

Does this class contain everything needed/used by the normal cron job? Should other functions also be called, eg preAction()?

[Serendesk]

See http://www.blesta.com/forums/index.php?/topic/5509-simple-service-to-avoid-cron/ for a solution.

[stacklicense]

The whole 5 minute cron thing is bizzare, why do orders submitted via the order form even need to wait for a cron job to fire...

Perhaps also look into implementing something similiar to hostbillapp.

https://hostbill.atlassian.net/wiki/spaces/DOCS/pages/491527/HostBill+Queue+Performance

[Paul]

On 8/23/2019 at 9:27 PM, stacklicense said:

The whole 5 minute cron thing is bizzare, why do orders submitted via the order form even need to wait for a cron job to fire...

Perhaps also look into implementing something similiar to hostbillapp.

https://hostbill.atlassian.net/wiki/spaces/DOCS/pages/491527/HostBill+Queue+Performance

• Customers may pay with a payment method that takes the client away from Blesta, which posts back to Blesta when the payment is completed.
• Provisioning services through certain modules may take a significant amount of time to complete, such that making the user wait in the browser may result in a timeout.

We are considering an enterprise level feature that adds a daemon to your server that runs continuously and processes certain jobs like order provisioning in near real-time.