Service Changes Failing Randomly

[Jonathan]

Service changes are failing at times, seemingly randomly.  There are no module logs corresponding to the time of the failure making me think it's happening before the module processing is attempted.

 

MariaDB [blesta]> select date_added,date_status from service_changes where status='error';
+---------------------+---------------------+
| date_added          | date_status         |
+---------------------+---------------------+
| 2015-06-07 19:44:48 | 2015-06-07 19:45:23 |
| 2015-06-08 13:44:15 | 2015-06-08 13:45:16 |
| 2015-06-08 15:39:00 | 2015-06-08 15:40:24 |
| 2015-06-08 18:07:25 | 2015-06-08 18:10:25 |
| 2015-06-08 16:08:54 | 2015-06-09 05:20:25 |
| 2015-06-08 15:04:27 | 2015-06-09 14:45:25 |
| 2015-06-09 05:08:01 | 2015-06-09 15:55:25 |
| 2015-06-09 05:07:15 | 2015-06-10 01:25:25 |
| 2015-06-08 13:44:55 | 2015-06-10 16:55:24 |
| 2015-06-10 17:54:17 | 2015-06-10 17:55:25 |
| 2015-06-11 08:53:43 | 2015-06-11 08:55:25 |
| 2015-06-12 01:07:57 | 2015-06-12 01:10:12 |
| 2015-06-12 20:49:23 | 2015-06-12 20:50:26 |
| 2015-06-14 05:06:43 | 2015-06-14 05:10:27 |
| 2015-06-14 18:28:22 | 2015-06-14 18:30:39 |
| 2015-06-15 15:52:36 | 2015-06-15 16:04:27 |
+---------------------+---------------------+

 

It's happening on brand new VPSs and older ones alike.  I've spot checked and on some of these payment was made before the first cron attempt so that rules out potential lack of payment resulting in an error.

[Tyson]

The service could be failing for any of several reasons, usually related to service validation at the time of processing. It would help for validation to occur before a service change is queued so that it can be refused due to error, but that is currently not possible. And there is not yet an interface from which to manage service changes, so I can only suggest to log the service errors in the meantime to identify the errors causing those changes to fail.

 

In the ServiceChanges model (/app/models/service_changes.php), find the reference to the service validation errors:

$errors = $this->Services->errors();

Below that add your logging (e.g. to a file).

 

I might simply add

echo "\nCHANGE #" . $service_change_id;
print_r($errors);

..then disable the Process Service Changes cron task from the Company Automation settings for a few minutes such that the task has not run within its set interval. Then re-enable the cron task and immediately run the cron manually from the System Automation settings before it is run via cron automatically. The paid pending service changes should be processed, and any errors should appear in the cron output. Note that you would need to update some records from the `service_changes` table from status "error" back to status "pending" to attempt to process them again.

[Jonathan]

I finally tied it to service validation.

 

 

a:1:{s:13:"configoptions";a:1:{s:5:"valid";s:70:"One of the configurable options selected is not valid for the service.";}}

 

I can edit the service, taking the approach of nuking all service_options rows related to the service_id and saving the manage service page fresh so I know for a fact there's good data there.  Set the service_changes.status back to pending and it fails again on the exact same error.

 

Saving without using pro-rata so it doesn't queue up works perfectly.

[Tyson]

That's interesting. How about disabling service change queuing via the "Queue Service Changes Until Paid" setting and then updating a service with prorate checked? This and the service queuing perform the same validation checks, so I would expect you to receive the same error.

 

That error is caused by one of the following:

1. The config option does not exist with the given value at the given pricing
2. The option is a quantity type and the given quantity is less than the minimum allowed
3. The option is a quantity type and the given quantity is more than the maximum allowed
4. The option is a quantity type and the given quantity does not follow the Step of allowed quantity values

If possible, it would be useful to compare the POST data of a successful edit (without queuing/proration) versus the queued data stored in `service_changes`.`data` of the same service to see what information is different.

[Jonathan]

I'm pretty sure I've ruled this down to a bug in the front-end theme (custom) which is rendering sliders as text input that's letting people input incorrect values.

[Tyson]

That would make sense.

[Michael]

I'm pretty sure I've ruled this down to a bug in the front-end theme (custom) which is rendering sliders as text input that's letting people input incorrect values.

 

is it the 3.2.0.js instead of 3.5.0.js I made that mistake before.

[Jonathan]

is it the 3.2.0.js instead of 3.5.0.js I made that mistake before.

 

Looks like they're both getting loaded but thanks for the recommendation.

[Tyson]

I'm going to close thread this as not a bug with Blesta. However, it would be useful to avoid this problem altogether by (CORE-1677) validating service fields before queuing them, which would be an improvement for sometime in the future.