Comments on: Still scratching my head. http://blog.xcski.com/2006/05/12/still-scratching-my-head Everything I used to bore people on newsgroups and mailing lists with, now in one inconvenient place. Fri, 05 Dec 2008 10:43:32 +0000 http://wordpress.org/?v=2.6.5 By: Paul Tomblin http://blog.xcski.com/2006/05/12/still-scratching-my-head#comment-3474 Paul Tomblin Sun, 14 May 2006 13:35:22 +0000 http://xcski.com/blogs/pt/2006/05/12/still-scratching-my-head#comment-3474 Richard, that's a good point. The job that's actually freezing is just doing queries, no updates, but there are other processes going on. But there are some other processes and one of them does some deletes from the same tables that this one is deleting. Also, there may be a "vacuum analyze;" going on at the same time. We're using PostgresSQL (as you might have guessed by the "vacuum analyze" thing). I'll have to look more closely into what else might be going on in that table, and how to monitor locks. I'm pretty sure we turned off autocommit, but we don't actually use transactions or explicit commits anywhere. I was thinking of doing so, but I haven't figured out the best way to do this - whether to make one big transaction around major activities like "the nightly expire run" or "handle this event", or small transactions around single operations like "delete this playlist and all the schedules that rely on it". Richard, that’s a good point. The job that’s actually freezing is just doing queries, no updates, but there are other processes going on. But there are some other processes and one of them does some deletes from the same tables that this one is deleting. Also, there may be a “vacuum analyze;” going on at the same time.

We’re using PostgresSQL (as you might have guessed by the “vacuum analyze” thing). I’ll have to look more closely into what else might be going on in that table, and how to monitor locks. I’m pretty sure we turned off autocommit, but we don’t actually use transactions or explicit commits anywhere. I was thinking of doing so, but I haven’t figured out the best way to do this - whether to make one big transaction around major activities like “the nightly expire run” or “handle this event”, or small transactions around single operations like “delete this playlist and all the schedules that rely on it”.

]]> By: Richard http://blog.xcski.com/2006/05/12/still-scratching-my-head#comment-3458 Richard Sun, 14 May 2006 08:09:51 +0000 http://xcski.com/blogs/pt/2006/05/12/still-scratching-my-head#comment-3458 What are your database changes like? if you've more than one insert or update in a job can they occur in different sequences? e.g. insert from tableA followed by a delete from tableB and another job task requires an insert into tableB followed by an insert into tableA? What are you using as a database? can you monitor its lock activity? What are your database changes like? if you’ve more than one insert or update in a job can they occur in different sequences? e.g. insert from tableA followed by a delete from tableB and another job task requires an insert into tableB followed by an insert into tableA?

What are you using as a database? can you monitor its lock activity?

]]> By: Tina Marie http://blog.xcski.com/2006/05/12/still-scratching-my-head#comment-3250 Tina Marie Fri, 12 May 2006 21:22:08 +0000 http://xcski.com/blogs/pt/2006/05/12/still-scratching-my-head#comment-3250 I vote for the job queue. If nothing else, create a thread pool, and feed it from the jobs. Everytime you get a job, check the threads. If one is idle, give them the job. If not, check to see if any are hung, kill it, then restart it and give it the job. And so on... We've found the best way to find lock problems is a multi-processor machine. We've got one or two quads that get loaned out just to debug this sort of thing. I vote for the job queue. If nothing else, create a thread pool, and feed it from the jobs. Everytime you get a job, check the threads. If one is idle, give them the job. If not, check to see if any are hung, kill it, then restart it and give it the job. And so on…

We’ve found the best way to find lock problems is a multi-processor machine. We’ve got one or two quads that get loaned out just to debug this sort of thing.

]]>