performance - Azure Queue delayed message -

-

i has strange behaviour on production deployment azure queue messages: of messages in queues appears big delay - minutes, , 10 minutes. befere ask setting delaytimeout when put message queue - not set delaytimeout message, message should appear immedeatly after placed in queue. @ moments not have big load. instances has no work load, , able process message fast, don't appear.

our service process millions of messages per month, able identify 10-50 messages processed big delay, fail sla in front of our customers.

does have idea can reason?

how overcome?

did faced similar issues?

some general ideas troubleshooting:

  1. are message queued processing - ie queue.addmessage operation returned , waiting 10 minutes - meaning can rule out client side retry policies etc being cause of problem.

  2. is there chance time calculation subject kind of clock skew problems. eg - if 1 of worker roles pulling messages has close out of sync other worker roles see this.

  3. is possible in situations message appearing delayed worker role responsible pulling messages failing or crashing. if client calls getmessage not respond appropriate acknowledgement within time specified invisibilitytimeout setting message become visible again queue service assumes client did not process message. tell if contributing factor looking @ dequeue count on these messages taking longer. more information can found here: http://msdn.microsoft.com/en-us/library/dd179474.aspx.

  4. is possible number of workers have pulling items queue insufficient @ times of day , delays caused queue being populated faster can pull messages queue.

  5. have enabled logging queues , looked see if can find specific operations (look @ e2elatency , serverlatency). http://blogs.msdn.com/b/windowsazurestorage/archive/tags/analytics+2d00+logging+_2600_amp_3b00_+metrics/. should enable client logging , try determine if client having connectivity problems , retry logic possibly kicking in.

and if none of these appear can please send me server logs (and ideally client side logs well) along account information (no passwords) jahogg @ microsoft dot com.

jason


Comments

Post a Comment

Popular posts from this blog

java - Intellij Synchronizing output directories .. -

-
i trying build project, run in debug mode , keep getting "synchronizing output directories" in intellij. takes around 5-10 minutes actual build. project empty , stored on local drive. tried invalidate catche has not worked. out of ideas .. had similar issues? most likely, of data on network drive or in profile being synchronized network drive. if on windows, try moving .ideaxxx/system directory , work directory somewhere else on c: drive.
Read more

git - Initial Commit: "fatal: could not create leading directories of ..." -

-
i trying make initial commit git repository on github unity project. followed along this guide am. note: reason or another, couldn't set unity's asset serialization mode force text, settled on mixed (which believe should work). when call git clone --bare . , error. note not creator of repository , contributor (though making initial commit). here's in terminal (i'm using git bash): welcome git (version 1.8.4-preview20130916) run 'git git' display index. run 'git <command>' display specific commands. cheddar@cheddar-pc ~ $ cd documents/ics168swarch/ cheddar@cheddar-pc ~/documents/ics168swarch $ git init initialized empty git repository in c:/users/cheddar/documents/ics168swarch/.git / cheddar@cheddar-pc ~/documents/ics168swarch (master) $ git add . warning: lf replaced crlf in assets/prefabs.meta. file have original line endings in working directory. warning: lf replaced crlf in assets/prefabs/pellet.prefab.meta. file have original
Read more