Ticket Piping Suddenly Stopped Working

[Virtovo]

Ticket downloads seem to have suddenly stopped working.  

 

I now get the error:

 

There are one or more cron tasks that have been executing for more than 60 minutes

 

In the automation section it does show that this task is still running (spinner next to the task).  It's been running for over two hours now.  Restarting the web-server does not appear to resolve the issue.

 

Not sure where to check for a log to explain what this task is doing.  The cron section in logs merely states this process started two hours ago.

[velaware]

Check this thread: http://www.blesta.com/forums/index.php?/topic/1052-system-status-there-are-one-or-more-cron-tasks-that-have-been-executing-for-more-than-60-minutes/

 

Specifically this post: http://www.blesta.com/forums/index.php?/topic/1052-system-status-there-are-one-or-more-cron-tasks-that-have-been-executing-for-more-than-60-minutes/?p=8567

 

I'm not aware of any other way to fix this right now.  But basically for some reason a cron task got stuck.  If you need any further help feel free to ask.

 

Basically though you need to have a way to run queries on your own (i.e.: by using phpMyAdmin or via the command line itself).

[velaware]

Oh and another thing to check is that the email server/service is running.  I don't use piping but could be the issue, don't know.

[Virtovo]

Thanks.  I've discovered the task which causes the problem as the interface leaves a spinner next to it.  I read in another thread that a reset button was to be included with 3.1.0.  I'm guessing this slipped.

 

For now then, I can either wait six hours or update the end time of that task?

[velaware]

Thanks.  I've discovered the task which causes the problem as the interface leaves a spinner next to it.  I read in another thread that a reset button was to be included with 3.1.0.  I'm guessing this slipped.

 

For now then, I can either wait six hours or update the end time of that task?

If you're going to update the end time, read this thread: http://www.blesta.com/forums/index.php?/topic/1805-there-are-one-or-more-cron-tasks-that-have-been-executing-for-more-than-60-minutes/ (post of interest: http://www.blesta.com/forums/index.php?/topic/1805-there-are-one-or-more-cron-tasks-that-have-been-executing-for-more-than-60-minutes/?p=13924 ).

 

I had the same issue as you just with IMAP instead of piping, and waiting 6 hours didn't do me any good.

[Virtovo]

If you're going to update the end time, read this thread: http://www.blesta.com/forums/index.php?/topic/1805-there-are-one-or-more-cron-tasks-that-have-been-executing-for-more-than-60-minutes/ (post of interest: http://www.blesta.com/forums/index.php?/topic/1805-there-are-one-or-more-cron-tasks-that-have-been-executing-for-more-than-60-minutes/?p=13924 ).

 

I had the same issue as you just with IMAP instead of piping, and waiting 6 hours didn't do me any good.

 

I managed to fix it by changing the end date.  Seems crazy that you need to run SQL queries on the database manually to fix this issue though!

[velaware]

I managed to fix it by changing the end date.  Seems crazy that you need to run SQL queries on the database manually to fix this issue though!

Agreed.  I'm thinking of writing a plugin actually to do this automatically.  Its pretty easy to do.

[Virtovo]

Agreed.  I'm thinking of writing a plugin actually to do this automatically.  Its pretty easy to do.

 

Actually made a mistake in the SQL query!  Now have 24000 items with wrong end date

 

Think it might be possible/safe to truncate the table?

[velaware]

Actually made a mistake in the SQL query!  Now have 24000 items with wrong end date

 

Think it might be possible/safe to truncate the table?

Personally I would advise against it but to be sure I would draw the attention of Paul, Cody or Tyson to this thread (PM them) so they're aware of this.  I don't want to advise you on something I know not much on (don't know what issues might occur if any).

[Virtovo]

Personally I would advise against it but to be sure I would draw the attention of Paul, Cody or Tyson to this thread (PM them) so they're aware of this.  I don't want to advise you on something I know not much on (don't know what issues might occur if any).

 

Thanks already checked with support.  Seems OK to delete entries older than a day otherwise there is a potential for tasks to run twice.

[Tyson]

Thanks.  I've discovered the task which causes the problem as the interface leaves a spinner next to it.  I read in another thread that a reset button was to be included with 3.1.0.  I'm guessing this slipped.

 

For now then, I can either wait six hours or update the end time of that task?

 

 

I managed to fix it by changing the end date.  Seems crazy that you need to run SQL queries on the database manually to fix this issue though!

 

A link to clear a task will appear next to it on the Automation page if Blesta determined that the task has stalled. It sounds like:

1. the task was still actually running
2. the cron had not yet finished running other tasks in the same group
3. or you had another issue that prevented the cron from finishing

Manually updating the database to clear this problem is not necessary, as you can do so through the interface, as long as you don't encounter one of the above 3 conditions.

[Virtovo]

A link to clear a task will appear next to it on the Automation page if Blesta determined that the task has stalled. It sounds like:

1. the task was still actually running
2. the cron had not yet finished running other tasks in the same group
3. or you had another issue that prevented the cron from finishing

Manually updating the database to clear this problem is not necessary, as you can do so through the interface, as long as you don't encounter one of the above 3 conditions.

 

Hello Tyson,

 

Thanks for the reply.  The task was running for a couple of hours (it normally completes within minutes).  Please advise what I should do in that situation

[Virtovo]

The above is even more pertinent as the task has yet again stalled for over an hour.  

[Tyson]

Since the issue is recurring, it sounds as if you have a larger issue, possibly an error occurring when the cron executes up to that point. Since you don't see the link to clear the task, you should delete this task from the log_cron table and run the cron manually through the admin interface.

 

You should be able to run the following query and then delete all of the results. It usually goes without saying, but you should backup the database before making manual changes.

SELECT * FROM `log_cron` WHERE `run_id` !=0 AND `end_date` IS NULL ORDER BY `start_date` DESC;

Then go to [settings] -> [system] -> [Automation] and run the cron manually. If there is an error, you should see it there, and then we can come up with a solution.

[Virtovo]

Since the issue is recurring, it sounds as if you have a larger issue, possibly an error occurring when the cron executes up to that point. Since you don't see the link to clear the task, you should delete this task from the log_cron table and run the cron manually through the admin interface.

 

You should be able to run the following query and then delete all of the results. It usually goes without saying, but you should backup the database before making manual changes.

SELECT * FROM `log_cron` WHERE `run_id` !=0 AND `end_date` IS NULL ORDER BY `start_date` DESC;

Then go to [settings] -> [system] -> [Automation] and run the cron manually. If there is an error, you should see it there, and then we can come up with a solution.

 

 

Sadly running the cron manually does not throw up the error.  The error does not happen every time the task runs and will run fine for hours (interval 5 minutes) then suddenly stop working.  Is there any way to get a log of what is going on during the POP download task?

[Tyson]

If you are logging cron output on your server, you can check the server logs to see what the error may be. Alternatively, when the task fails to run, you could email in a ticket yourself, delete the task from the database as I described above, and then run the cron manually through the admin interface. Assuming that having a ticket ready to be imported causes the issue, you should be able to see the error in the admin interface. Also, you could probably have your cronjob setup to write to a file (or email you) rather than go to /dev/null, so you can view the output from the cron.

 

Since you don't see the link on the company Automation page to clear the task, the task is failing, and causing the rest of the company tasks to fail, which sounds like a fatal error. Blesta can't log fatal errors, so you'll need to look outside of Blesta for that information.

[Virtovo]

Running the task manually immediately after clearing the database works fine and tickets are imported

[Virtovo]

To update, I notice the email provider we use has experienced intermittent downtime of service; however I am unsure if this is the cause as I have no log of the ticket import to see itself.  Even if this is the case, should the task not simply fail instead of stalling?

[Tyson]

To update, I notice the email provider we use has experienced intermittent downtime of service; however I am unsure if this is the cause as I have no log of the ticket import to see itself.  Even if this is the case, should the task not simply fail instead of stalling?

 

If the service has been down intermittently, then the task could still be running, waiting for a response. I imagine it would have eventually timed out. It's also possible that the task could have experienced a fatal error only when run in CLI mode automatically when the cron runs. In the case of the latter, you can pipe the cron output to a file on your server and then take a look at it if you experience this problem again to see if there are any errors.

[Virtovo]

If the service has been down intermittently, then the task could still be running, waiting for a response. I imagine it would have eventually timed out. It's also possible that the task could have experienced a fatal error only when run in CLI mode automatically when the cron runs. In the case of the latter, you can pipe the cron output to a file on your server and then take a look at it if you experience this problem again to see if there are any errors.

 

Right, the problem has occured again.  I've been piping cron out.  The best I get out of cron (which isn't informative at all) is:

 

Attempting plugin cron for support_manager poll_tickets.

 

 

After that it stops running the task but all others run with no problem.  E.g:

 

Attempting to apply credits to open invoices.

There are no invoices to which credits may be applied.

The apply credits task has completed.

Attempting to deliver invoices scheduled for delivery.

No invoices are scheduled to be delivered.

The deliver invoices task has completed.

Attempting to provision paid pending services.

The paid pending services task has completed.

Attempting to unsuspend paid suspended services.

The unsuspend services task has completed.

Attempting to process renewing services.

The process renewing services task has completed.

Attempting plugin cron for order accept_paid_orders.

Finished plugin cron for order accept_paid_orders.

All tasks have been completed.

Attempting to run all system tasks.

All system tasks have been completed.

 

 

I'll ask again, is there any way to get more detailed logs of what is going on with the support plugin when it is polling tickets?