Discussion:
[zeromq-dev] SEHException 0x80004005 from ZeroMQ/libzmq
Aaron Friesen
2016-11-09 23:10:11 UTC
Permalink
All,

Getting an SEHException 0x80004005 from ZeroMQ (4.1.0.21) / libzmq (4.1.5.0)

Multiple processes went down with the same exception at the same time.

Was not able to get a dump but the application logs showed the following stack trace:

System.Exception System.Runtime.InteropServices.SEHException (0x80004005): External component has thrown an exception.
at ZeroMQ.lib.zmq.zmq_msg_send(IntPtr msg, IntPtr socket, Int32 flags)
at ZeroMQ.ZSocket.SendFrame(ZFrame frame, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, Int32& sent, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg)
at xxxxxx.SocketsThread(Object eventWaitHandle)

No line numbers available, but based on the logged message, it would have occurred in the following code. Because the stack trace does not include any of the calls within the try block (PollIn, ProcessRequest, ProcessSubscription), I am at a loss as to what exactly was executing at the time of the exception that was calling SendMessage.

Does anyone have any ideas as to what I might be doing wrong, or what the problem might be and how to avoid it?



ZSocket[] sockets = new ZSocket[] { _requestSocket, _subscriberSocket };
ZPollItem[] pollItems = new ZPollItem[] { ZPollItem.CreateReceiver(), ZPollItem.CreateReceiver() };
ZMessage[] messages = null;

try
{
TimeSpan timeout = TimeSpan.FromMilliseconds(100);

while (_run)
{
if (ZPollItems.PollIn(sockets, pollItems, out messages, out error, timeout))
{
if (error == ZError.EAGAIN)
continue;

if (error == ZError.ETERM)
break;

if (messages == null)
continue;

if (messages[0] != null) // Request
ProcessRequest(messages[0]);

if (messages[1] != null) // Subscription
ProcessSubscription(messages[1]);
}
else
{
if (error == ZError.EAGAIN)
continue;

if (error != ZError.None)
break;
}
}
}
catch (Exception ex)
{
if (!(ex is ThreadAbortException))
{
_logger.FatalException(string.Format("Exception encountered while polling for messages on sockets. Thread '{0}' shutting down.", threadName), ex);

Environment.Exit(-1);
}
}

Thank you in advance,

Aaron Friesen
Aaron Friesen
2016-11-14 19:30:15 UTC
Permalink
All,

Getting an SEHException 0x80004005 from ZeroMQ (4.1.0.21) / libzmq (4.1.5.0)

Multiple processes went down with the same exception at the same time.

Was not able to get a dump but the application logs showed the following stack trace:

System.Exception System.Runtime.InteropServices.SEHException (0x80004005): External component has thrown an exception.
at ZeroMQ.lib.zmq.zmq_msg_send(IntPtr msg, IntPtr socket, Int32 flags)
at ZeroMQ.ZSocket.SendFrame(ZFrame frame, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, Int32& sent, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg)
at xxxxxx.SocketsThread(Object eventWaitHandle)

No line numbers available, but based on the logged message, it would have occurred in the following code. Because the stack trace does not include any of the calls within the try block (PollIn, ProcessRequest, ProcessSubscription), I am at a loss as to what exactly was executing at the time of the exception that was calling SendMessage.

Does anyone have any ideas as to what I might be doing wrong, or what the problem might be and how to avoid it?



ZSocket[] sockets = new ZSocket[] { _requestSocket, _subscriberSocket };
ZPollItem[] pollItems = new ZPollItem[] { ZPollItem.CreateReceiver(), ZPollItem.CreateReceiver() };
ZMessage[] messages = null;

try
{
TimeSpan timeout = TimeSpan.FromMilliseconds(100);

while (_run)
{
if (ZPollItems.PollIn(sockets, pollItems, out messages, out error, timeout))
{
if (error == ZError.EAGAIN)
continue;

if (error == ZError.ETERM)
break;

if (messages == null)
continue;

if (messages[0] != null) // Request
ProcessRequest(messages[0]);

if (messages[1] != null) // Subscription
ProcessSubscription(messages[1]);
}
else
{
if (error == ZError.EAGAIN)
continue;

if (error != ZError.None)
break;
}
}
}
catch (Exception ex)
{
if (!(ex is ThreadAbortException))
{
_logger.FatalException(string.Format("Exception encountered while polling for messages on sockets. Thread '{0}' shutting down.", threadName), ex);

Environment.Exit(-1);
}
}

Thank you in advance,

Aaron Friesen
Osiris Pedroso
2016-11-23 10:38:32 UTC
Permalink
I know how to generate minidumps in Windows and create a small (~20Kb) file
that would have a snapshot of the stack and lots of other goodies.
To access it, one opens the generated .DMP file with "WinDBG.exe -k
minidump.dmp", enter the command ".ecxr" then "kvn" to see stack at the
failure point in time.
If PDB symbol files for the correct version of the DLLs being used are
available at their build locations, the "kvn" command will even tell you
the source file and line number where the exception happened and you will
be able to see local variable values for the functions on the stack by
typing "dv" in WinDBG for each stack frame.

Obviously this is Windows only functionality.

Earlier in the year I made a contribution to documentation of ZeroMQ using
a .DOC file and I felt shunned by the community when my PR was denied
because it used a Windows document format.
I even offered to format the same file as PDF, because the important thing
was the information it contained, not the format, to no avail.
At the time, (and even now) it felt to me like a betrayal of the C4 tenets,
but lets not get into religious wars here.

I can tell you that in my professional work, I have brought down my
company's main product from 10 crashes/DAY/user to 0.5 crashes/MONTH/user
by iterating on generating these minidumps, asking the users to send them
in, analyzing them and fixing the problems they brought to our attention,
so this is proven technology.

If the community feels that this is a good idea (add a Windows specific
code to generate minidumps when exceptions happen), I would love to invest
the time to this effort.
Post by Aaron Friesen
All,
Getting an SEHException 0x80004005 from ZeroMQ (4.1.0.21) / libzmq (4.1.5.0)
Multiple processes went down with the same exception at the same time.
External component has thrown an exception.
at ZeroMQ.lib.zmq.zmq_msg_send(IntPtr msg, IntPtr socket, Int32 flags)
at ZeroMQ.ZSocket.SendFrame(ZFrame frame, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, Int32& sent,
ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg)
at xxxxxx.SocketsThread(Object eventWaitHandle)
No line numbers available, but based on the logged message, it would have
occurred in the following code. Because the stack trace does not include
any of the calls within the try block (PollIn, ProcessRequest,
ProcessSubscription), I am at a loss as to what exactly was executing at
the time of the exception that was calling SendMessage.
Does anyone have any ideas as to what I might be doing wrong, or what the
problem might be and how to avoid it?
ZSocket[] sockets = new ZSocket[] { _requestSocket, _subscriberSocket };
ZPollItem[] pollItems = new ZPollItem[] { ZPollItem.CreateReceiver(),
ZPollItem.CreateReceiver() };
ZMessage[] messages = null;
try
{
TimeSpan timeout = TimeSpan.FromMilliseconds(100);
while (_run)
{
if (ZPollItems.PollIn(sockets, pollItems, out
messages, out error, timeout))
{
if (error == ZError.EAGAIN)
continue;
if (error == ZError.ETERM)
break;
if (messages == null)
continue;
if (messages[0] != null) // Request
ProcessRequest(messages[0]);
if (messages[1] != null) // Subscription
ProcessSubscription(messages[1]);
}
else
{
if (error == ZError.EAGAIN)
continue;
if (error != ZError.None)
break;
}
}
}
catch (Exception ex)
{
if (!(ex is ThreadAbortException))
{
_logger.FatalException(string.Format("Exception
encountered while polling for messages on sockets. Thread '{0}' shutting
down.", threadName), ex);
Environment.Exit(-1);
}
}
Thank you in advance,
Aaron Friesen
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Auer, Jens
2016-11-23 13:16:31 UTC
Permalink
Hi,

I had a similar experience when I included crashrpt to a Windows application I was working on. It greatly increases the ability to fix bugs, discuss priorities and fight the “the software is getting worse with every release” flames from other people.

However, I don’t think this should be included in ZeroMQ. ZeroMQ is a library that application developer use, and the same developer must decide if a crash reporter should be used or not.

Cheers,
Jens
--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154
***@cgi.com<mailto:***@cgi.com>
Unsere Pflichtangaben gemÀß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.

CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to CGI Group Inc. and its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for delivery of this message to such person), or you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply e-mail.

From: zeromq-dev [mailto:zeromq-dev-***@lists.zeromq.org] On Behalf Of Osiris Pedroso
Sent: 23 November 2016 11:39
To: zeromq-***@lists.zeromq.org
Subject: Re: [zeromq-dev] SEHException 0x80004005 from ZeroMQ/libzmq

I know how to generate minidumps in Windows and create a small (~20Kb) file that would have a snapshot of the stack and lots of other goodies.
To access it, one opens the generated .DMP file with "WinDBG.exe -k minidump.dmp", enter the command ".ecxr" then "kvn" to see stack at the failure point in time.
If PDB symbol files for the correct version of the DLLs being used are available at their build locations, the "kvn" command will even tell you the source file and line number where the exception happened and you will be able to see local variable values for the functions on the stack by typing "dv" in WinDBG for each stack frame.

Obviously this is Windows only functionality.

Earlier in the year I made a contribution to documentation of ZeroMQ using a .DOC file and I felt shunned by the community when my PR was denied because it used a Windows document format.
I even offered to format the same file as PDF, because the important thing was the information it contained, not the format, to no avail.
At the time, (and even now) it felt to me like a betrayal of the C4 tenets, but lets not get into religious wars here.

I can tell you that in my professional work, I have brought down my company's main product from 10 crashes/DAY/user to 0.5 crashes/MONTH/user by iterating on generating these minidumps, asking the users to send them in, analyzing them and fixing the problems they brought to our attention, so this is proven technology.

If the community feels that this is a good idea (add a Windows specific code to generate minidumps when exceptions happen), I would love to invest the time to this effort.


On Mon, Nov 14, 2016 at 1:30 PM Aaron Friesen <***@spirae.com<mailto:***@spirae.com>> wrote:
All,

Getting an SEHException 0x80004005 from ZeroMQ (4.1.0.21) / libzmq (4.1.5.0)

Multiple processes went down with the same exception at the same time.

Was not able to get a dump but the application logs showed the following stack trace:

System.Exception System.Runtime.InteropServices.SEHException (0x80004005): External component has thrown an exception.
at ZeroMQ.lib.zmq.zmq_msg_send(IntPtr msg, IntPtr socket, Int32 flags)
at ZeroMQ.ZSocket.SendFrame(ZFrame frame, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, Int32& sent, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg)
at xxxxxx.SocketsThread(Object eventWaitHandle)

No line numbers available, but based on the logged message, it would have occurred in the following code. Because the stack trace does not include any of the calls within the try block (PollIn, ProcessRequest, ProcessSubscription), I am at a loss as to what exactly was executing at the time of the exception that was calling SendMessage.

Does anyone have any ideas as to what I might be doing wrong, or what the problem might be and how to avoid it?



ZSocket[] sockets = new ZSocket[] { _requestSocket, _subscriberSocket };
ZPollItem[] pollItems = new ZPollItem[] { ZPollItem.CreateReceiver(), ZPollItem.CreateReceiver() };
ZMessage[] messages = null;

try
{
TimeSpan timeout = TimeSpan.FromMilliseconds(100);

while (_run)
{
if (ZPollItems.PollIn(sockets, pollItems, out messages, out error, timeout))
{
if (error == ZError.EAGAIN)
continue;

if (error == ZError.ETERM)
break;

if (messages == null)
continue;

if (messages[0] != null) // Request
ProcessRequest(messages[0]);

if (messages[1] != null) // Subscription
ProcessSubscription(messages[1]);
}
else
{
if (error == ZError.EAGAIN)
continue;

if (error != ZError.None)
break;
}
}
}
catch (Exception ex)
{
if (!(ex is ThreadAbortException))
{
_logger.FatalException(string.Format("Exception encountered while polling for messages on sockets. Thread '{0}' shutting down.", threadName), ex);

Environment.Exit(-1);
}
}

Thank you in advance,

Aaron Friesen

_______________________________________________
zeromq-dev mailing list
zeromq-***@lists.zeromq.org<mailto:zeromq-***@lists.zeromq.org>
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Auer, Jens
2016-11-24 08:40:02 UTC
Permalink
Hi,

Thinking about it, I may change my mind. It may be interesting to be able to generate a core dump (mini dump) in the zmq_assert statements if it can be switched on or off, e.g. by a define or even context property. I think this would help finding issues in ZeroMQ.

Cheers,
Jens
--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154
***@cgi.com<mailto:***@cgi.com>
Unsere Pflichtangaben gemÀß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.

CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to CGI Group Inc. and its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for delivery of this message to such person), or you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply e-mail.

From: zeromq-dev [mailto:zeromq-dev-***@lists.zeromq.org] On Behalf Of Auer, Jens
Sent: 23 November 2016 14:17
To: ZeroMQ development list
Subject: Re: [zeromq-dev] SEHException 0x80004005 from ZeroMQ/libzmq

Hi,

I had a similar experience when I included crashrpt to a Windows application I was working on. It greatly increases the ability to fix bugs, discuss priorities and fight the “the software is getting worse with every release” flames from other people.

However, I don’t think this should be included in ZeroMQ. ZeroMQ is a library that application developer use, and the same developer must decide if a crash reporter should be used or not.

Cheers,
Jens
--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154
***@cgi.com<mailto:***@cgi.com>
Unsere Pflichtangaben gemÀß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.

CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to CGI Group Inc. and its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for delivery of this message to such person), or you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply e-mail.

From: zeromq-dev [mailto:zeromq-dev-***@lists.zeromq.org] On Behalf Of Osiris Pedroso
Sent: 23 November 2016 11:39
To: zeromq-***@lists.zeromq.org<mailto:zeromq-***@lists.zeromq.org>
Subject: Re: [zeromq-dev] SEHException 0x80004005 from ZeroMQ/libzmq

I know how to generate minidumps in Windows and create a small (~20Kb) file that would have a snapshot of the stack and lots of other goodies.
To access it, one opens the generated .DMP file with "WinDBG.exe -k minidump.dmp", enter the command ".ecxr" then "kvn" to see stack at the failure point in time.
If PDB symbol files for the correct version of the DLLs being used are available at their build locations, the "kvn" command will even tell you the source file and line number where the exception happened and you will be able to see local variable values for the functions on the stack by typing "dv" in WinDBG for each stack frame.

Obviously this is Windows only functionality.

Earlier in the year I made a contribution to documentation of ZeroMQ using a .DOC file and I felt shunned by the community when my PR was denied because it used a Windows document format.
I even offered to format the same file as PDF, because the important thing was the information it contained, not the format, to no avail.
At the time, (and even now) it felt to me like a betrayal of the C4 tenets, but lets not get into religious wars here.

I can tell you that in my professional work, I have brought down my company's main product from 10 crashes/DAY/user to 0.5 crashes/MONTH/user by iterating on generating these minidumps, asking the users to send them in, analyzing them and fixing the problems they brought to our attention, so this is proven technology.

If the community feels that this is a good idea (add a Windows specific code to generate minidumps when exceptions happen), I would love to invest the time to this effort.


On Mon, Nov 14, 2016 at 1:30 PM Aaron Friesen <***@spirae.com<mailto:***@spirae.com>> wrote:
All,

Getting an SEHException 0x80004005 from ZeroMQ (4.1.0.21) / libzmq (4.1.5.0)

Multiple processes went down with the same exception at the same time.

Was not able to get a dump but the application logs showed the following stack trace:

System.Exception System.Runtime.InteropServices.SEHException (0x80004005): External component has thrown an exception.
at ZeroMQ.lib.zmq.zmq_msg_send(IntPtr msg, IntPtr socket, Int32 flags)
at ZeroMQ.ZSocket.SendFrame(ZFrame frame, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, Int32& sent, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg)
at xxxxxx.SocketsThread(Object eventWaitHandle)

No line numbers available, but based on the logged message, it would have occurred in the following code. Because the stack trace does not include any of the calls within the try block (PollIn, ProcessRequest, ProcessSubscription), I am at a loss as to what exactly was executing at the time of the exception that was calling SendMessage.

Does anyone have any ideas as to what I might be doing wrong, or what the problem might be and how to avoid it?



ZSocket[] sockets = new ZSocket[] { _requestSocket, _subscriberSocket };
ZPollItem[] pollItems = new ZPollItem[] { ZPollItem.CreateReceiver(), ZPollItem.CreateReceiver() };
ZMessage[] messages = null;

try
{
TimeSpan timeout = TimeSpan.FromMilliseconds(100);

while (_run)
{
if (ZPollItems.PollIn(sockets, pollItems, out messages, out error, timeout))
{
if (error == ZError.EAGAIN)
continue;

if (error == ZError.ETERM)
break;

if (messages == null)
continue;

if (messages[0] != null) // Request
ProcessRequest(messages[0]);

if (messages[1] != null) // Subscription
ProcessSubscription(messages[1]);
}
else
{
if (error == ZError.EAGAIN)
continue;

if (error != ZError.None)
break;
}
}
}
catch (Exception ex)
{
if (!(ex is ThreadAbortException))
{
_logger.FatalException(string.Format("Exception encountered while polling for messages on sockets. Thread '{0}' shutting down.", threadName), ex);

Environment.Exit(-1);
}
}

Thank you in advance,

Aaron Friesen

_______________________________________________
zeromq-dev mailing list
zeromq-***@lists.zeromq.org<mailto:zeromq-***@lists.zeromq.org>
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Luca Boccassi
2016-11-24 11:37:02 UTC
Permalink
Hi,

I have done something along those lines a while back for *nix. If
libunwind is available at build time then this code will print a
gdb-style backtrace on abort:

https://github.com/zeromq/libzmq/blob/master/src/err.cpp#L391

Need to be careful with core dumps and backtraces as they might expose
private data about proprietary applications, so it should be opt-in to
avoid nasty surprises for enterprise users and the like.

But I don't see any problem in principle, as long as it's a
well-documented opt-in.
Post by Auer, Jens
Hi,
Thinking about it, I may change my mind. It may be interesting to be able to generate a core dump (mini dump) in the zmq_assert statements if it can be switched on or off, e.g. by a define or even context property. I think this would help finding issues in ZeroMQ.
Cheers,
Jens
--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154
Unsere Pflichtangaben gemÀß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.
CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to CGI Group Inc. and its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for delivery of this message to such person), or you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply e-mail.
Sent: 23 November 2016 14:17
To: ZeroMQ development list
Subject: Re: [zeromq-dev] SEHException 0x80004005 from ZeroMQ/libzmq
Hi,
I had a similar experience when I included crashrpt to a Windows application I was working on. It greatly increases the ability to fix bugs, discuss priorities and fight the “the software is getting worse with every release” flames from other people.
However, I don’t think this should be included in ZeroMQ. ZeroMQ is a library that application developer use, and the same developer must decide if a crash reporter should be used or not.
Cheers,
Jens
--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154
Unsere Pflichtangaben gemÀß § 35a GmbHG / §§ 161, 125a HGB finden Sie unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.
CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging to CGI Group Inc. and its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for delivery of this message to such person), or you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply e-mail.
Sent: 23 November 2016 11:39
Subject: Re: [zeromq-dev] SEHException 0x80004005 from ZeroMQ/libzmq
I know how to generate minidumps in Windows and create a small (~20Kb) file that would have a snapshot of the stack and lots of other goodies.
To access it, one opens the generated .DMP file with "WinDBG.exe -k minidump.dmp", enter the command ".ecxr" then "kvn" to see stack at the failure point in time.
If PDB symbol files for the correct version of the DLLs being used are available at their build locations, the "kvn" command will even tell you the source file and line number where the exception happened and you will be able to see local variable values for the functions on the stack by typing "dv" in WinDBG for each stack frame.
Obviously this is Windows only functionality.
Earlier in the year I made a contribution to documentation of ZeroMQ using a .DOC file and I felt shunned by the community when my PR was denied because it used a Windows document format.
I even offered to format the same file as PDF, because the important thing was the information it contained, not the format, to no avail.
At the time, (and even now) it felt to me like a betrayal of the C4 tenets, but lets not get into religious wars here.
I can tell you that in my professional work, I have brought down my company's main product from 10 crashes/DAY/user to 0.5 crashes/MONTH/user by iterating on generating these minidumps, asking the users to send them in, analyzing them and fixing the problems they brought to our attention, so this is proven technology.
If the community feels that this is a good idea (add a Windows specific code to generate minidumps when exceptions happen), I would love to invest the time to this effort.
All,
Getting an SEHException 0x80004005 from ZeroMQ (4.1.0.21) / libzmq (4.1.5.0)
Multiple processes went down with the same exception at the same time.
System.Exception System.Runtime.InteropServices.SEHException (0x80004005): External component has thrown an exception.
at ZeroMQ.lib.zmq.zmq_msg_send(IntPtr msg, IntPtr socket, Int32 flags)
at ZeroMQ.ZSocket.SendFrame(ZFrame frame, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, Int32& sent, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags, ZError& error)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg)
at xxxxxx.SocketsThread(Object eventWaitHandle)
No line numbers available, but based on the logged message, it would have occurred in the following code. Because the stack trace does not include any of the calls within the try block (PollIn, ProcessRequest, ProcessSubscription), I am at a loss as to what exactly was executing at the time of the exception that was calling SendMessage.
Does anyone have any ideas as to what I might be doing wrong, or what the problem might be and how to avoid it?
ZSocket[] sockets = new ZSocket[] { _requestSocket, _subscriberSocket };
ZPollItem[] pollItems = new ZPollItem[] { ZPollItem.CreateReceiver(), ZPollItem.CreateReceiver() };
ZMessage[] messages = null;
try
{
TimeSpan timeout = TimeSpan.FromMilliseconds(100);
while (_run)
{
if (ZPollItems.PollIn(sockets, pollItems, out messages, out error, timeout))
{
if (error == ZError.EAGAIN)
continue;
if (error == ZError.ETERM)
break;
if (messages == null)
continue;
if (messages[0] != null) // Request
ProcessRequest(messages[0]);
if (messages[1] != null) // Subscription
ProcessSubscription(messages[1]);
}
else
{
if (error == ZError.EAGAIN)
continue;
if (error != ZError.None)
break;
}
}
}
catch (Exception ex)
{
if (!(ex is ThreadAbortException))
{
_logger.FatalException(string.Format("Exception encountered while polling for messages on sockets. Thread '{0}' shutting down.", threadName), ex);
Environment.Exit(-1);
}
}
Thank you in advance,
Aaron Friesen
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
--
Kind regards,
Luca Boccassi
Osiris Pedroso
2016-12-04 01:04:39 UTC
Permalink
I thought some more about this and I think I would like to implement this
functionality as an actor named zdump.

Implementing it as an actor would allow any programs that use CZMQ actors
to add the ability to save minidumps on exceptions, and they would only be
active if the developer intended and created an instance of zdump in the
process.

So while instantiated, it would hook itself as a the handler of last resort
in the C Run Time library.
When this actor's instance is destroyed it removes itself, therefore no
more minidumps would be created.
The messages that this actor would accept would be:
"ENABLE" "DEFAULT", specific exception code (e.g. 0xc0000005 is access
violation, 0x80004005 is access denied,...)
"DISABLE" "DEFAULT", specific exception code
"WHERE" "root-dir-where-to-save-dumps\", default "%TEMP%"
"ROOTNAME" "DUMP", default current's executable rootname

DEFAULT could be a name for access violation exception code, since this is
the most common and important exception that ones wants to know is
happening in their programs. Depending how the program is written, it could
be swallowed silently by implementation. Creating the minidumps would let
the developer/user know that is taking place, plus save a the exact context
where it happened.

Multiple instances can be created, since handler of last resort can be
chained (defining one returns the function pointer to the previous one).

I don't know if it should be part of CZMQ or just an object out there that
people could use.
Obviously as part of CZMQ many more people would know about it and use it.
But writing those XML files to generate interface and code are
complicated...

Toughts?
Post by Luca Boccassi
Hi,
I have done something along those lines a while back for *nix. If
libunwind is available at build time then this code will print a
https://github.com/zeromq/libzmq/blob/master/src/err.cpp#L391
Need to be careful with core dumps and backtraces as they might expose
private data about proprietary applications, so it should be opt-in to
avoid nasty surprises for enterprise users and the like.
But I don't see any problem in principle, as long as it's a
well-documented opt-in.
Post by Auer, Jens
Hi,
Thinking about it, I may change my mind. It may be interesting to be
able to generate a core dump (mini dump) in the zmq_assert statements if it
can be switched on or off, e.g. by a define or even context property. I
think this would help finding issues in ZeroMQ.
Post by Auer, Jens
Cheers,
Jens
--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154 <+49%206151%2036860154>
Unsere Pflichtangaben gemÀß § 35a GmbHG / §§ 161, 125a HGB finden Sie
unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.
Post by Auer, Jens
CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging
to CGI Group Inc. and its affiliates may be contained in this message. If
you are not a recipient indicated or intended in this message (or
responsible for delivery of this message to such person), or you think for
any reason that this message may have been addressed to you in error, you
may not use or copy or deliver this message to anyone else. In such case,
you should destroy this message and are asked to notify the sender by reply
e-mail.
Of Auer, Jens
Post by Auer, Jens
Sent: 23 November 2016 14:17
To: ZeroMQ development list
Subject: Re: [zeromq-dev] SEHException 0x80004005 from ZeroMQ/libzmq
Hi,
I had a similar experience when I included crashrpt to a Windows
application I was working on. It greatly increases the ability to fix bugs,
discuss priorities and fight the “the software is getting worse with every
release” flames from other people.
Post by Auer, Jens
However, I don’t think this should be included in ZeroMQ. ZeroMQ is a
library that application developer use, and the same developer must decide
if a crash reporter should be used or not.
Post by Auer, Jens
Cheers,
Jens
--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154 <+49%206151%2036860154>
Unsere Pflichtangaben gemÀß § 35a GmbHG / §§ 161, 125a HGB finden Sie
unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.
Post by Auer, Jens
CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging
to CGI Group Inc. and its affiliates may be contained in this message. If
you are not a recipient indicated or intended in this message (or
responsible for delivery of this message to such person), or you think for
any reason that this message may have been addressed to you in error, you
may not use or copy or deliver this message to anyone else. In such case,
you should destroy this message and are asked to notify the sender by reply
e-mail.
Of Osiris Pedroso
Post by Auer, Jens
Sent: 23 November 2016 11:39
Subject: Re: [zeromq-dev] SEHException 0x80004005 from ZeroMQ/libzmq
I know how to generate minidumps in Windows and create a small (~20Kb)
file that would have a snapshot of the stack and lots of other goodies.
Post by Auer, Jens
To access it, one opens the generated .DMP file with "WinDBG.exe -k
minidump.dmp", enter the command ".ecxr" then "kvn" to see stack at the
failure point in time.
Post by Auer, Jens
If PDB symbol files for the correct version of the DLLs being used are
available at their build locations, the "kvn" command will even tell you
the source file and line number where the exception happened and you will
be able to see local variable values for the functions on the stack by
typing "dv" in WinDBG for each stack frame.
Post by Auer, Jens
Obviously this is Windows only functionality.
Earlier in the year I made a contribution to documentation of ZeroMQ
using a .DOC file and I felt shunned by the community when my PR was denied
because it used a Windows document format.
Post by Auer, Jens
I even offered to format the same file as PDF, because the important
thing was the information it contained, not the format, to no avail.
Post by Auer, Jens
At the time, (and even now) it felt to me like a betrayal of the C4
tenets, but lets not get into religious wars here.
Post by Auer, Jens
I can tell you that in my professional work, I have brought down my
company's main product from 10 crashes/DAY/user to 0.5 crashes/MONTH/user
by iterating on generating these minidumps, asking the users to send them
in, analyzing them and fixing the problems they brought to our attention,
so this is proven technology.
Post by Auer, Jens
If the community feels that this is a good idea (add a Windows specific
code to generate minidumps when exceptions happen), I would love to invest
the time to this effort.
Post by Auer, Jens
All,
Getting an SEHException 0x80004005 from ZeroMQ (4.1.0.21) / libzmq
(4.1.5.0)
Post by Auer, Jens
Multiple processes went down with the same exception at the same time.
Was not able to get a dump but the application logs showed the following
System.Exception System.Runtime.InteropServices.SEHException
(0x80004005): External component has thrown an exception.
Post by Auer, Jens
at ZeroMQ.lib.zmq.zmq_msg_send(IntPtr msg, IntPtr socket, Int32 flags)
at ZeroMQ.ZSocket.SendFrame(ZFrame frame, ZSocketFlags flags, ZError&
error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, Int32& sent,
ZSocketFlags flags, ZError& error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, ZSocketFlags flags,
ZError& error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags, ZError&
error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg)
at xxxxxx.SocketsThread(Object eventWaitHandle)
No line numbers available, but based on the logged message, it would
have occurred in the following code. Because the stack trace does not
include any of the calls within the try block (PollIn, ProcessRequest,
ProcessSubscription), I am at a loss as to what exactly was executing at
the time of the exception that was calling SendMessage.
Post by Auer, Jens
Does anyone have any ideas as to what I might be doing wrong, or what
the problem might be and how to avoid it?
Post by Auer, Jens
ZSocket[] sockets = new ZSocket[] { _requestSocket,
_subscriberSocket };
Post by Auer, Jens
ZPollItem[] pollItems = new ZPollItem[] {
ZPollItem.CreateReceiver(), ZPollItem.CreateReceiver() };
Post by Auer, Jens
ZMessage[] messages = null;
try
{
TimeSpan timeout = TimeSpan.FromMilliseconds(100);
while (_run)
{
if (ZPollItems.PollIn(sockets, pollItems, out
messages, out error, timeout))
Post by Auer, Jens
{
if (error == ZError.EAGAIN)
continue;
if (error == ZError.ETERM)
break;
if (messages == null)
continue;
if (messages[0] != null) // Request
ProcessRequest(messages[0]);
if (messages[1] != null) // Subscription
ProcessSubscription(messages[1]);
}
else
{
if (error == ZError.EAGAIN)
continue;
if (error != ZError.None)
break;
}
}
}
catch (Exception ex)
{
if (!(ex is ThreadAbortException))
{
_logger.FatalException(string.Format("Exception
encountered while polling for messages on sockets. Thread '{0}' shutting
down.", threadName), ex);
Post by Auer, Jens
Environment.Exit(-1);
}
}
Thank you in advance,
Aaron Friesen
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
--
Kind regards,
Luca Boccassi
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Osiris Pedroso
2016-12-04 01:18:21 UTC
Permalink
Quick stack at zdump.h included
Post by Osiris Pedroso
I thought some more about this and I think I would like to implement this
functionality as an actor named zdump.
Implementing it as an actor would allow any programs that use CZMQ actors
to add the ability to save minidumps on exceptions, and they would only be
active if the developer intended and created an instance of zdump in the
process.
So while instantiated, it would hook itself as a the handler of last
resort in the C Run Time library.
When this actor's instance is destroyed it removes itself, therefore no
more minidumps would be created.
"ENABLE" "DEFAULT", specific exception code (e.g. 0xc0000005 is access
violation, 0x80004005 is access denied,...)
"DISABLE" "DEFAULT", specific exception code
"WHERE" "root-dir-where-to-save-dumps\", default "%TEMP%"
"ROOTNAME" "DUMP", default current's executable rootname
DEFAULT could be a name for access violation exception code, since this is
the most common and important exception that ones wants to know is
happening in their programs. Depending how the program is written, it could
be swallowed silently by implementation. Creating the minidumps would let
the developer/user know that is taking place, plus save a the exact context
where it happened.
Multiple instances can be created, since handler of last resort can be
chained (defining one returns the function pointer to the previous one).
I don't know if it should be part of CZMQ or just an object out there that
people could use.
Obviously as part of CZMQ many more people would know about it and use it.
But writing those XML files to generate interface and code are
complicated...
Toughts?
Hi,
I have done something along those lines a while back for *nix. If
libunwind is available at build time then this code will print a
https://github.com/zeromq/libzmq/blob/master/src/err.cpp#L391
Need to be careful with core dumps and backtraces as they might expose
private data about proprietary applications, so it should be opt-in to
avoid nasty surprises for enterprise users and the like.
But I don't see any problem in principle, as long as it's a
well-documented opt-in.
Post by Auer, Jens
Hi,
Thinking about it, I may change my mind. It may be interesting to be
able to generate a core dump (mini dump) in the zmq_assert statements if it
can be switched on or off, e.g. by a define or even context property. I
think this would help finding issues in ZeroMQ.
Post by Auer, Jens
Cheers,
Jens
--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154 <+49%206151%2036860154>
Unsere Pflichtangaben gemÀß § 35a GmbHG / §§ 161, 125a HGB finden Sie
unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.
Post by Auer, Jens
CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging
to CGI Group Inc. and its affiliates may be contained in this message. If
you are not a recipient indicated or intended in this message (or
responsible for delivery of this message to such person), or you think for
any reason that this message may have been addressed to you in error, you
may not use or copy or deliver this message to anyone else. In such case,
you should destroy this message and are asked to notify the sender by reply
e-mail.
Of Auer, Jens
Post by Auer, Jens
Sent: 23 November 2016 14:17
To: ZeroMQ development list
Subject: Re: [zeromq-dev] SEHException 0x80004005 from ZeroMQ/libzmq
Hi,
I had a similar experience when I included crashrpt to a Windows
application I was working on. It greatly increases the ability to fix bugs,
discuss priorities and fight the “the software is getting worse with every
release” flames from other people.
Post by Auer, Jens
However, I don’t think this should be included in ZeroMQ. ZeroMQ is a
library that application developer use, and the same developer must decide
if a crash reporter should be used or not.
Post by Auer, Jens
Cheers,
Jens
--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154 <+49%206151%2036860154>
Unsere Pflichtangaben gemÀß § 35a GmbHG / §§ 161, 125a HGB finden Sie
unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.
Post by Auer, Jens
CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging
to CGI Group Inc. and its affiliates may be contained in this message. If
you are not a recipient indicated or intended in this message (or
responsible for delivery of this message to such person), or you think for
any reason that this message may have been addressed to you in error, you
may not use or copy or deliver this message to anyone else. In such case,
you should destroy this message and are asked to notify the sender by reply
e-mail.
Of Osiris Pedroso
Post by Auer, Jens
Sent: 23 November 2016 11:39
Subject: Re: [zeromq-dev] SEHException 0x80004005 from ZeroMQ/libzmq
I know how to generate minidumps in Windows and create a small (~20Kb)
file that would have a snapshot of the stack and lots of other goodies.
Post by Auer, Jens
To access it, one opens the generated .DMP file with "WinDBG.exe -k
minidump.dmp", enter the command ".ecxr" then "kvn" to see stack at the
failure point in time.
Post by Auer, Jens
If PDB symbol files for the correct version of the DLLs being used are
available at their build locations, the "kvn" command will even tell you
the source file and line number where the exception happened and you will
be able to see local variable values for the functions on the stack by
typing "dv" in WinDBG for each stack frame.
Post by Auer, Jens
Obviously this is Windows only functionality.
Earlier in the year I made a contribution to documentation of ZeroMQ
using a .DOC file and I felt shunned by the community when my PR was denied
because it used a Windows document format.
Post by Auer, Jens
I even offered to format the same file as PDF, because the important
thing was the information it contained, not the format, to no avail.
Post by Auer, Jens
At the time, (and even now) it felt to me like a betrayal of the C4
tenets, but lets not get into religious wars here.
Post by Auer, Jens
I can tell you that in my professional work, I have brought down my
company's main product from 10 crashes/DAY/user to 0.5 crashes/MONTH/user
by iterating on generating these minidumps, asking the users to send them
in, analyzing them and fixing the problems they brought to our attention,
so this is proven technology.
Post by Auer, Jens
If the community feels that this is a good idea (add a Windows specific
code to generate minidumps when exceptions happen), I would love to invest
the time to this effort.
Post by Auer, Jens
All,
Getting an SEHException 0x80004005 from ZeroMQ (4.1.0.21) / libzmq
(4.1.5.0)
Post by Auer, Jens
Multiple processes went down with the same exception at the same time.
Was not able to get a dump but the application logs showed the following
System.Exception System.Runtime.InteropServices.SEHException
(0x80004005): External component has thrown an exception.
Post by Auer, Jens
at ZeroMQ.lib.zmq.zmq_msg_send(IntPtr msg, IntPtr socket, Int32 flags)
at ZeroMQ.ZSocket.SendFrame(ZFrame frame, ZSocketFlags flags, ZError&
error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, Int32& sent,
ZSocketFlags flags, ZError& error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, ZSocketFlags flags,
ZError& error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags, ZError&
error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg)
at xxxxxx.SocketsThread(Object eventWaitHandle)
No line numbers available, but based on the logged message, it would
have occurred in the following code. Because the stack trace does not
include any of the calls within the try block (PollIn, ProcessRequest,
ProcessSubscription), I am at a loss as to what exactly was executing at
the time of the exception that was calling SendMessage.
Post by Auer, Jens
Does anyone have any ideas as to what I might be doing wrong, or what
the problem might be and how to avoid it?
Post by Auer, Jens
ZSocket[] sockets = new ZSocket[] { _requestSocket,
_subscriberSocket };
Post by Auer, Jens
ZPollItem[] pollItems = new ZPollItem[] {
ZPollItem.CreateReceiver(), ZPollItem.CreateReceiver() };
Post by Auer, Jens
ZMessage[] messages = null;
try
{
TimeSpan timeout = TimeSpan.FromMilliseconds(100);
while (_run)
{
if (ZPollItems.PollIn(sockets, pollItems, out
messages, out error, timeout))
Post by Auer, Jens
{
if (error == ZError.EAGAIN)
continue;
if (error == ZError.ETERM)
break;
if (messages == null)
continue;
if (messages[0] != null) // Request
ProcessRequest(messages[0]);
if (messages[1] != null) // Subscription
ProcessSubscription(messages[1]);
}
else
{
if (error == ZError.EAGAIN)
continue;
if (error != ZError.None)
break;
}
}
}
catch (Exception ex)
{
if (!(ex is ThreadAbortException))
{
_logger.FatalException(string.Format("Exception
encountered while polling for messages on sockets. Thread '{0}' shutting
down.", threadName), ex);
Post by Auer, Jens
Environment.Exit(-1);
}
}
Thank you in advance,
Aaron Friesen
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
--
Kind regards,
Luca Boccassi
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Osiris Pedroso
2016-12-04 01:19:22 UTC
Permalink
I meant quick stab at zdump.h...
Post by Osiris Pedroso
Quick stack at zdump.h included
I thought some more about this and I think I would like to implement this
functionality as an actor named zdump.
Implementing it as an actor would allow any programs that use CZMQ actors
to add the ability to save minidumps on exceptions, and they would only be
active if the developer intended and created an instance of zdump in the
process.
So while instantiated, it would hook itself as a the handler of last
resort in the C Run Time library.
When this actor's instance is destroyed it removes itself, therefore no
more minidumps would be created.
"ENABLE" "DEFAULT", specific exception code (e.g. 0xc0000005 is access
violation, 0x80004005 is access denied,...)
"DISABLE" "DEFAULT", specific exception code
"WHERE" "root-dir-where-to-save-dumps\", default "%TEMP%"
"ROOTNAME" "DUMP", default current's executable rootname
DEFAULT could be a name for access violation exception code, since this is
the most common and important exception that ones wants to know is
happening in their programs. Depending how the program is written, it could
be swallowed silently by implementation. Creating the minidumps would let
the developer/user know that is taking place, plus save a the exact context
where it happened.
Multiple instances can be created, since handler of last resort can be
chained (defining one returns the function pointer to the previous one).
I don't know if it should be part of CZMQ or just an object out there that
people could use.
Obviously as part of CZMQ many more people would know about it and use it.
But writing those XML files to generate interface and code are
complicated...
Toughts?
Hi,
I have done something along those lines a while back for *nix. If
libunwind is available at build time then this code will print a
https://github.com/zeromq/libzmq/blob/master/src/err.cpp#L391
Need to be careful with core dumps and backtraces as they might expose
private data about proprietary applications, so it should be opt-in to
avoid nasty surprises for enterprise users and the like.
But I don't see any problem in principle, as long as it's a
well-documented opt-in.
Post by Auer, Jens
Hi,
Thinking about it, I may change my mind. It may be interesting to be
able to generate a core dump (mini dump) in the zmq_assert statements if it
can be switched on or off, e.g. by a define or even context property. I
think this would help finding issues in ZeroMQ.
Post by Auer, Jens
Cheers,
Jens
--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154 <+49%206151%2036860154>
Unsere Pflichtangaben gemÀß § 35a GmbHG / §§ 161, 125a HGB finden Sie
unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.
Post by Auer, Jens
CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging
to CGI Group Inc. and its affiliates may be contained in this message. If
you are not a recipient indicated or intended in this message (or
responsible for delivery of this message to such person), or you think for
any reason that this message may have been addressed to you in error, you
may not use or copy or deliver this message to anyone else. In such case,
you should destroy this message and are asked to notify the sender by reply
e-mail.
Of Auer, Jens
Post by Auer, Jens
Sent: 23 November 2016 14:17
To: ZeroMQ development list
Subject: Re: [zeromq-dev] SEHException 0x80004005 from ZeroMQ/libzmq
Hi,
I had a similar experience when I included crashrpt to a Windows
application I was working on. It greatly increases the ability to fix bugs,
discuss priorities and fight the “the software is getting worse with every
release” flames from other people.
Post by Auer, Jens
However, I don’t think this should be included in ZeroMQ. ZeroMQ is a
library that application developer use, and the same developer must decide
if a crash reporter should be used or not.
Post by Auer, Jens
Cheers,
Jens
--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154 <+49%206151%2036860154>
Unsere Pflichtangaben gemÀß § 35a GmbHG / §§ 161, 125a HGB finden Sie
unter de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.
Post by Auer, Jens
CONFIDENTIALITY NOTICE: Proprietary/Confidential information belonging
to CGI Group Inc. and its affiliates may be contained in this message. If
you are not a recipient indicated or intended in this message (or
responsible for delivery of this message to such person), or you think for
any reason that this message may have been addressed to you in error, you
may not use or copy or deliver this message to anyone else. In such case,
you should destroy this message and are asked to notify the sender by reply
e-mail.
Of Osiris Pedroso
Post by Auer, Jens
Sent: 23 November 2016 11:39
Subject: Re: [zeromq-dev] SEHException 0x80004005 from ZeroMQ/libzmq
I know how to generate minidumps in Windows and create a small (~20Kb)
file that would have a snapshot of the stack and lots of other goodies.
Post by Auer, Jens
To access it, one opens the generated .DMP file with "WinDBG.exe -k
minidump.dmp", enter the command ".ecxr" then "kvn" to see stack at the
failure point in time.
Post by Auer, Jens
If PDB symbol files for the correct version of the DLLs being used are
available at their build locations, the "kvn" command will even tell you
the source file and line number where the exception happened and you will
be able to see local variable values for the functions on the stack by
typing "dv" in WinDBG for each stack frame.
Post by Auer, Jens
Obviously this is Windows only functionality.
Earlier in the year I made a contribution to documentation of ZeroMQ
using a .DOC file and I felt shunned by the community when my PR was denied
because it used a Windows document format.
Post by Auer, Jens
I even offered to format the same file as PDF, because the important
thing was the information it contained, not the format, to no avail.
Post by Auer, Jens
At the time, (and even now) it felt to me like a betrayal of the C4
tenets, but lets not get into religious wars here.
Post by Auer, Jens
I can tell you that in my professional work, I have brought down my
company's main product from 10 crashes/DAY/user to 0.5 crashes/MONTH/user
by iterating on generating these minidumps, asking the users to send them
in, analyzing them and fixing the problems they brought to our attention,
so this is proven technology.
Post by Auer, Jens
If the community feels that this is a good idea (add a Windows specific
code to generate minidumps when exceptions happen), I would love to invest
the time to this effort.
Post by Auer, Jens
All,
Getting an SEHException 0x80004005 from ZeroMQ (4.1.0.21) / libzmq
(4.1.5.0)
Post by Auer, Jens
Multiple processes went down with the same exception at the same time.
Was not able to get a dump but the application logs showed the following
System.Exception System.Runtime.InteropServices.SEHException
(0x80004005): External component has thrown an exception.
Post by Auer, Jens
at ZeroMQ.lib.zmq.zmq_msg_send(IntPtr msg, IntPtr socket, Int32 flags)
at ZeroMQ.ZSocket.SendFrame(ZFrame frame, ZSocketFlags flags, ZError&
error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, Int32& sent,
ZSocketFlags flags, ZError& error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1 frames, ZSocketFlags flags,
ZError& error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags, ZError&
error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendMessage(ZMessage msg, ZSocketFlags flags)
at ZeroMQ.ZSocket.SendMessage(ZMessage msg)
at xxxxxx.SocketsThread(Object eventWaitHandle)
No line numbers available, but based on the logged message, it would
have occurred in the following code. Because the stack trace does not
include any of the calls within the try block (PollIn, ProcessRequest,
ProcessSubscription), I am at a loss as to what exactly was executing at
the time of the exception that was calling SendMessage.
Post by Auer, Jens
Does anyone have any ideas as to what I might be doing wrong, or what
the problem might be and how to avoid it?
Post by Auer, Jens
ZSocket[] sockets = new ZSocket[] { _requestSocket,
_subscriberSocket };
Post by Auer, Jens
ZPollItem[] pollItems = new ZPollItem[] {
ZPollItem.CreateReceiver(), ZPollItem.CreateReceiver() };
Post by Auer, Jens
ZMessage[] messages = null;
try
{
TimeSpan timeout = TimeSpan.FromMilliseconds(100);
while (_run)
{
if (ZPollItems.PollIn(sockets, pollItems, out
messages, out error, timeout))
Post by Auer, Jens
{
if (error == ZError.EAGAIN)
continue;
if (error == ZError.ETERM)
break;
if (messages == null)
continue;
if (messages[0] != null) // Request
ProcessRequest(messages[0]);
if (messages[1] != null) // Subscription
ProcessSubscription(messages[1]);
}
else
{
if (error == ZError.EAGAIN)
continue;
if (error != ZError.None)
break;
}
}
}
catch (Exception ex)
{
if (!(ex is ThreadAbortException))
{
_logger.FatalException(string.Format("Exception
encountered while polling for messages on sockets. Thread '{0}' shutting
down.", threadName), ex);
Post by Auer, Jens
Environment.Exit(-1);
}
}
Thank you in advance,
Aaron Friesen
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
--
Kind regards,
Luca Boccassi
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Luca Boccassi
2016-12-04 13:20:22 UTC
Permalink
This sounds like a good idea! If you could model it in a .api and mark
it as draft we can merge it and iterate from there
Post by Osiris Pedroso
I meant quick stab at zdump.h...
Quick stack at zdump.h included
On Sat, Dec 3, 2016 at 7:04 PM Osiris Pedroso
I thought some more about this and I think I would
like to implement this functionality as an actor named
zdump.
Implementing it as an actor would allow any programs
that use CZMQ actors to add the ability to save
minidumps on exceptions, and they would only be active
if the developer intended and created an instance of
zdump in the process.
So while instantiated, it would hook itself as a the
handler of last resort in the C Run Time library.
When this actor's instance is destroyed it removes
itself, therefore no more minidumps would be created.
"ENABLE" "DEFAULT", specific exception code (e.g.
0xc0000005 is access violation, 0x80004005 is access
denied,...)
"DISABLE" "DEFAULT", specific exception code
"WHERE" "root-dir-where-to-save-dumps\", default "%
TEMP%"
"ROOTNAME" "DUMP", default current's executable rootname
DEFAULT could be a name for access violation exception
code, since this is the most common and important
exception that ones wants to know is happening in
their programs. Depending how the program is written,
it could be swallowed silently by implementation.
Creating the minidumps would let the developer/user
know that is taking place, plus save a the exact
context where it happened.
Multiple instances can be created, since handler of
last resort can be chained (defining one returns the
function pointer to the previous one).
I don't know if it should be part of CZMQ or just an
object out there that people could use.
Obviously as part of CZMQ many more people would know
about it and use it.
But writing those XML files to generate interface and
code are complicated...
Toughts?
On Thu, Nov 24, 2016 at 5:37 AM Luca Boccassi
Hi,
I have done something along those lines a
while back for *nix. If
libunwind is available at build time then this
code will print a
https://github.com/zeromq/libzmq/blob/master/src/err.cpp#L391
Need to be careful with core dumps and
backtraces as they might expose
private data about proprietary applications,
so it should be opt-in to
avoid nasty surprises for enterprise users and
the like.
But I don't see any problem in principle, as
long as it's a
well-documented opt-in.
On Thu, 2016-11-24 at 08:40 +0000, Auer, Jens
Post by Auer, Jens
Hi,
Thinking about it, I may change my mind. It
may be interesting to be able to generate a
core dump (mini dump) in the zmq_assert
statements if it can be switched on or off,
e.g. by a define or even context property. I
think this would help finding issues in
ZeroMQ.
Post by Auer, Jens
Cheers,
Jens
--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154
Unsere Pflichtangaben gemÀß § 35a GmbHG / §§
161, 125a HGB finden Sie unter
de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.
Proprietary/Confidential information belonging
to CGI Group Inc. and its affiliates may be
contained in this message. If you are not a
recipient indicated or intended in this
message (or responsible for delivery of this
message to such person), or you think for any
reason that this message may have been
addressed to you in error, you may not use or
copy or deliver this message to anyone else.
In such case, you should destroy this message
and are asked to notify the sender by reply
e-mail.
Post by Auer, Jens
From: zeromq-dev
On Behalf Of Auer, Jens
Post by Auer, Jens
Sent: 23 November 2016 14:17
To: ZeroMQ development list
Subject: Re: [zeromq-dev] SEHException
0x80004005 from ZeroMQ/libzmq
Post by Auer, Jens
Hi,
I had a similar experience when I included
crashrpt to a Windows application I was
working on. It greatly increases the ability
to fix bugs, discuss priorities and fight the
“the software is getting worse with every
release” flames from other people.
Post by Auer, Jens
However, I don’t think this should be
included in ZeroMQ. ZeroMQ is a library that
application developer use, and the same
developer must decide if a crash reporter
should be used or not.
Post by Auer, Jens
Cheers,
Jens
--
Dr. Jens Auer | CGI | Software Engineer
CGI Deutschland Ltd. & Co. KG
Rheinstraße 95 | 64295 Darmstadt | Germany
T: +49 6151 36860 154
Unsere Pflichtangaben gemÀß § 35a GmbHG / §§
161, 125a HGB finden Sie unter
de.cgi.com/pflichtangaben<http://de.cgi.com/pflichtangaben>.
Proprietary/Confidential information belonging
to CGI Group Inc. and its affiliates may be
contained in this message. If you are not a
recipient indicated or intended in this
message (or responsible for delivery of this
message to such person), or you think for any
reason that this message may have been
addressed to you in error, you may not use or
copy or deliver this message to anyone else.
In such case, you should destroy this message
and are asked to notify the sender by reply
e-mail.
Post by Auer, Jens
From: zeromq-dev
On Behalf Of Osiris Pedroso
Post by Auer, Jens
Sent: 23 November 2016 11:39
Subject: Re: [zeromq-dev] SEHException
0x80004005 from ZeroMQ/libzmq
Post by Auer, Jens
I know how to generate minidumps in Windows
and create a small (~20Kb) file that would
have a snapshot of the stack and lots of other
goodies.
Post by Auer, Jens
To access it, one opens the generated .DMP
file with "WinDBG.exe -k minidump.dmp", enter
the command ".ecxr" then "kvn" to see stack at
the failure point in time.
Post by Auer, Jens
If PDB symbol files for the correct version
of the DLLs being used are available at their
build locations, the "kvn" command will even
tell you the source file and line number where
the exception happened and you will be able to
see local variable values for the functions on
the stack by typing "dv" in WinDBG for each
stack frame.
Post by Auer, Jens
Obviously this is Windows only
functionality.
Post by Auer, Jens
Earlier in the year I made a contribution to
documentation of ZeroMQ using a .DOC file and
I felt shunned by the community when my PR was
denied because it used a Windows document
format.
Post by Auer, Jens
I even offered to format the same file as
PDF, because the important thing was the
information it contained, not the format, to
no avail.
Post by Auer, Jens
At the time, (and even now) it felt to me
like a betrayal of the C4 tenets, but lets not
get into religious wars here.
Post by Auer, Jens
I can tell you that in my professional work,
I have brought down my company's main product
from 10 crashes/DAY/user to 0.5
crashes/MONTH/user by iterating on generating
these minidumps, asking the users to send them
in, analyzing them and fixing the problems
they brought to our attention, so this is
proven technology.
Post by Auer, Jens
If the community feels that this is a good
idea (add a Windows specific code to generate
minidumps when exceptions happen), I would
love to invest the time to this effort.
Post by Auer, Jens
On Mon, Nov 14, 2016 at 1:30 PM Aaron
Friesen
Post by Auer, Jens
All,
Getting an SEHException 0x80004005 from
ZeroMQ (4.1.0.21) / libzmq (4.1.5.0)
Post by Auer, Jens
Multiple processes went down with the same
exception at the same time.
Post by Auer, Jens
Was not able to get a dump but the
application logs showed the following stack
Post by Auer, Jens
System.Exception
System.Runtime.InteropServices.SEHException
(0x80004005): External component has thrown an
exception.
Post by Auer, Jens
at ZeroMQ.lib.zmq.zmq_msg_send(IntPtr msg,
IntPtr socket, Int32 flags)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendFrame(ZFrame frame,
ZSocketFlags flags, ZError& error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1
frames, Int32& sent, ZSocketFlags flags,
ZError& error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendFrames(IEnumerable`1
frames, ZSocketFlags flags, ZError& error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendMessage(ZMessage msg,
ZSocketFlags flags, ZError& error)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendMessage(ZMessage msg,
ZSocketFlags flags)
Post by Auer, Jens
at ZeroMQ.ZSocket.SendMessage(ZMessage msg)
at xxxxxx.SocketsThread(Object
eventWaitHandle)
Post by Auer, Jens
No line numbers available, but based on the
logged message, it would have occurred in the
following code. Because the stack trace does
not include any of the calls within the try
block (PollIn, ProcessRequest,
ProcessSubscription), I am at a loss as to
what exactly was executing at the time of the
exception that was calling SendMessage.
Post by Auer, Jens
Does anyone have any ideas as to what I
might be doing wrong, or what the problem
might be and how to avoid it?
Post by Auer, Jens
ZSocket[] sockets = new
ZSocket[] { _requestSocket,
_subscriberSocket };
Post by Auer, Jens
ZPollItem[] pollItems = new
ZPollItem[] { ZPollItem.CreateReceiver(),
ZPollItem.CreateReceiver() };
Post by Auer, Jens
ZMessage[] messages = null;
try
{
TimeSpan timeout =
TimeSpan.FromMilliseconds(100);
Post by Auer, Jens
while (_run)
{
if
(ZPollItems.PollIn(sockets, pollItems, out
messages, out error, timeout))
Post by Auer, Jens
{
if (error ==
ZError.EAGAIN)
Post by Auer, Jens
continue;
if (error ==
ZError.ETERM)
Post by Auer, Jens
break;
if (messages ==
null)
Post by Auer, Jens
continue;
if
(messages[0] != null) // Request
ProcessRequest(messages[0]);
Post by Auer, Jens
if
(messages[1] != null) // Subscription
ProcessSubscription(messages[1]);
Post by Auer, Jens
}
else
{
if (error ==
ZError.EAGAIN)
Post by Auer, Jens
continue;
if (error !=
ZError.None)
Post by Auer, Jens
break;
}
}
}
catch (Exception ex)
{
if (!(ex is
ThreadAbortException))
Post by Auer, Jens
{
_logger.FatalException(string.Format("Exception encountered while polling for messages on sockets. Thread '{0}' shutting down.", threadName), ex);
Environment.Exit(-1);
Post by Auer, Jens
}
}
Thank you in advance,
Aaron Friesen
_______________________________________________
Post by Auer, Jens
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________
Post by Auer, Jens
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
--
Kind regards,
Luca Boccassi
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________
zeromq-dev mailing list
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Loading...