Just a popular stackoverflow question & answer which was deleted as not
constructive even with 10+ score and thousands of views (by a mod, so
can’t vote to revert - thanks!) Keep in mind that it’s from 2009 without
later reviews.
Question:
I am searching for a tool that tests
SIP calls. A
platform that makes a call from SIP device A to SIP device B and reports results…
Any idea? A simulation platform would be ideal.
My answer:
There are many solutions. Some more broken than others. Here’s a quick
summary of what I’ve found while looking for a base for a proper
automated testing solution.
It’s ok if you want only a single dialog at a time. What doesn’t work
here is complex scenarios where you need to synchronise 2 call legs, do
registration, call and presence in the same scenario. If you go this
way, you’ll end up with running multiple sipp scenarios for each
conversation element separately. Sipp also doesn’t scale at all for
media transfers. Even though it’s multithreaded, something stops it from
running concurrently - if you look at htop for example, you’ll see
that sipp never crosses the 100% line. Around 50 media calls it starts
to cut audio and take all CPU of the machine.
It can sometimes lose track of what’s happening, some packets which
don’t even belong to the call really can result in a failed test. It’s
got some silly bugs too, like case-sensitive comparing of the headers.
Ruby-based solution where you have to write your own scenarios in Ruby.
It’s got its own SIP stack and lots of tests. While it’s generally good
and handles a lot of complex scenarios nicely, its design is terrible in
my opinion. Bugs are hard to track down and after a week I had >10
patches that I needed just to make it do basic stuff. Later I learned
that some of the scenarios should have been written in a slightly
different way, but SIPr developers were not really responsive and it
took a lot of time to find out about it. There was no good documentation
either. Synchronising actions of many agents is a hard problem, since
they’d rather use an event-based, but still single-threaded approach…
it just makes you concentrate too much on “what order can this happen in
and do I handle it correctly” and worrying about sync/async actions,
rather than writing the actual test.
Commercial solution. Never tested it properly since the basic
functionality is missing from the evaluation version and it’s hard to
spend that much money on something you’re not sure works…
Java-based solution reusing Jain-SIP stack. It can do almost any
scenario and is fairly good. It tries to make everything non-blocking /
action based leading to a similar situation SIPr has, but in this case
it’s trivial to make it parallel / threaded so it’s much easier to deal
with. It has its own share of bugs, so not everything works well in the
vanilla package, but most of the stuff is patchable. The developers seem
to be busy with other projects, so it’s not updated for a long time. If
you need transfers, presence, dialog-info, custom messages, RTP
handling, etc. - you’ll have to write your own modifications to support
them. It is not good for performance testing though - each session on
the test client side is rather heavy compared to the typical server side
processing. That means you’ll most likely need a number of hosts doing
the testing per one server/proxy.
If you’re a Java-hater like me, it can be used in a simple way from
Jython, JRuby or any other JVM language.
In the end, I chose SIPunit as the least broken/evil/unusable solution.
It is by no means perfect, but… it works in most cases. If I was doing
the project once again with all this knowledge, I’d probably reuse SIPp
configurations and try to write my own solution that uses proper
threading - but I guesstimate it’s at least a ½ year project for one
person, to make it good enough for production.