Tcp throws ArgumentNullException, when receiving data (mono, akka 1.3.1)

[blackclavus]

This exception thrown when a client (non-akka) send data to server (akka) via Tcp. Note that this only happen if code runs on Mono.

System.ArgumentNullException: The message cannot be null.
Parameter name: message
  at Akka.Actor.Envelope..ctor (System.Object message, Akka.Actor.IActorRef sender, Akka.Actor.ActorSystem system) [0x00032] in <388272c00ff54cb2808e25c3aa633b3f>:0
  at Akka.Actor.ActorCell.SendMessage (Akka.Actor.IActorRef sender, System.Object message) [0x00009] in <388272c00ff54cb2808e25c3aa633b3f>:0
  at Akka.Actor.LocalActorRef.TellInternal (System.Object message, Akka.Actor.IActorRef sender) [0x00000] in <388272c00ff54cb2808e25c3aa633b3f>:0
  at Akka.Actor.ActorRefBase.Tell (System.Object message, Akka.Actor.IActorRef sender) [0x0000a] in <388272c00ff54cb2808e25c3aa633b3f>:0
  at Akka.Actor.ActorRefImplicitSenderExtensions.Tell (Akka.Actor.IActorRef receiver, System.Object message) [0x00006] in <388272c00ff54cb2808e25c3aa633b3f>:0
  at Akka.IO.TcpExt.OnComplete (System.Object sender, System.Net.Sockets.SocketAsyncEventArgs e) [0x00016] in <388272c00ff54cb2808e25c3aa633b3f>:0
  at System.Net.Sockets.SocketAsyncEventArgs.OnCompleted (System.Net.Sockets.SocketAsyncEventArgs e) [0x0000e] in <a67b90c5acf54694896b770f716b945d>:0
  at System.Net.Sockets.SocketAsyncEventArgs.Complete () [0x00000] in <a67b90c5acf54694896b770f716b945d>:0
  at System.Net.Sockets.Socket+<>c.<.cctor>b__306_7 (System.IAsyncResult ares) [0x0005d] in <a67b90c5acf54694896b770f716b945d>:0
  at System.Net.Sockets.SocketAsyncResult+<>c.<Complete>b__27_0 (System.Object state) [0x0000b] in <a67b90c5acf54694896b770f716b945d>:0
  at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00008] in <902ab9e386384bec9c07fa19aa938869>:0
  at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in <902ab9e386384bec9c07fa19aa938869>:0
  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <902ab9e386384bec9c07fa19aa938869>:0

Sample Code:

class Program
{
	static void Main(string[] args)
	{
		using (var system = ActorSystem.Create("tcp-test", ConfigurationFactory.ParseString(@"
			akka.suppress-json-serializer-warning = on
			akka.loglevel = DEBUG")))
		{
			IPEndPoint serverAddress = new IPEndPoint(IPAddress.Any, 9000);


			system.ActorOf(Props.Create(() => new TestActor(serverAddress)));
			Console.ReadLine();
		}
	}

	class TestActor : ReceiveActor
	{
		private readonly ILoggingAdapter _log = Context.GetLogger();
		private readonly EndPoint _address;

		public IActorRef _tcp;

		public TestActor(EndPoint endPoint)
		{
			_address = endPoint;

			Receive<Tcp.Bound>(bound => _log.Debug("Tcp success to bound to: {0}", bound.LocalAddress));

			Receive<Tcp.Connected>(connected =>
			{
				_log.Debug("Remote address {0} connected", connected.RemoteAddress);
				_tcp = Sender;
				
				Sender.Tell(new Tcp.Register(Self));
				Sender.Tell(Tcp.Write.Create(ByteString.FromString("Hello!")));
			});

			Receive<Tcp.Received>(received =>
			{
				_log.Debug("Received: {0}", received.Data.ToString());
			});
		}

		protected override void PreStart()
		{
			Context.System.Tcp().Tell(new Tcp.Bind(Self, _address, 100));
		}
	}
}

Client Code

class Program
{
	static void Main(string[] args)
	{
		var tcp = new TcpClient();
		Console.Write("Connect To:");
		string line;
		bool connected = false;
		do
		{
			line = Console.ReadLine();
			if (!connected)
			{
				tcp.Connect(line, 9000);
				connected = true;
			}
			else
			{
				tcp.Send(Encoding.ASCII.GetBytes(line));
			}
		} while (line != "exit");
	}
}

note: I'm using .net framework 4.6 (on both server and client)

[Horusiath]

@blackclavus I think, we can say that Akka.IO support for Mono is a "best effort" right now. It's caused mostly by fact, that the current support for things like buffers and SocketAsyncEventArgs (and socket API in general) on Mono is horrible.

The reason behind this bug is also part of internal bug of Mono itself and I'm afraid we cannot do much with it at this point.

[mmoussikhine]

I have encountered the same problem (both in 1.3.1 and in dev under the Mono 5.2.0.215 on Ubuntu) and tracked it to the Akka.IO.OnComplete().

It calls ResolveMessage(e) to get the message, however under Mono this method always returns null. The culprit is the [MethodImpl(MethodImplOptions.AggressiveInlining)] annotation:

[MethodImpl(MethodImplOptions.AggressiveInlining)] 
private static Tcp.SocketCompleted ResolveMessage(SocketAsyncEventArgs e)
{
  ...
}

Without this optimization, ResolveMessage() is working fine.

More information:

• Running application in Debug mode (for example in Rider) does not trigger the bug.
• It always fails when application is in Run mode either in Debug or Release build.
• Adding a result variable like:

[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static Tcp.SocketCompleted ResolveMessage(SocketAsyncEventArgs e)
{
    Tcp.SocketCompleted res;
    switch (e.LastOperation)
    {
        case SocketAsyncOperation.Receive:
        case SocketAsyncOperation.ReceiveFrom:
        case SocketAsyncOperation.ReceiveMessageFrom:
            res = Tcp.SocketReceived.Instance;
            break;
        case SocketAsyncOperation.Send:
        case SocketAsyncOperation.SendTo:
        case SocketAsyncOperation.SendPackets:
            res = Tcp.SocketSent.Instance;
            break;
        case SocketAsyncOperation.Accept:
            res = Tcp.SocketAccepted.Instance;
            break;
        case SocketAsyncOperation.Connect:
            res = Tcp.SocketConnected.Instance;
            break;
        default:
            throw new NotSupportedException($"Socket operation {e.LastOperation} is not supported");
    }
    return res;
}

solves the problem, but I guess the result is the same as simply removing the annotation (optimization is ignored).

This is certainly a Mono problem. I am hesitant to suggest that it should be fixed in Akka, since it is clearly not Akka bug. But, would you be able to contact Mono team then, since you would have better understanding of what is really going on here?

Is it also reasonable to assume that similar issues could manifest themselves in other cases using the same optimization?

Meanwhile, one can disable mono inline optimization to at least be able run the code:

mono -O=all,-inline ...

[Aaronontheweb]

@mmoussikhine nice detective work! Yep, we can report the Mono bug here: https://bugzilla.xamarin.com/

I'd recommend filing a bug there and linking to this Github issue for reference.

In the meantime, IMHO, I'd recommend disabling the optimization on Akka just so we can get our Mono users up and running again. Yes, there will be a performance hit - but we can put the optimization back once Mono runtime is patched with the latest.

[Horusiath]

@mmoussikhine Would you mind sending a PR?

[mmoussikhine]

Sure. Just to disable the optimization on this method?

[Aaronontheweb]

@mmoussikhine yep, and report the issue on Mono if you don't mind :D

[mmoussikhine]

Mono bug report:
https://bugzilla.xamarin.com/show_bug.cgi?id=59608

[Aaronontheweb]

thanks @mmoussikhine

[Aaronontheweb]

You sent in a PR earlier as well right, removing the optimization?

[mmoussikhine]

Here: #3115

[Aaronontheweb]

Closed via #3115