A Minetest peer initiates a connection by sending a packet with an invalid peer_id, for whatever reason the code for doing this ran on both the client and the server meaning you could connect to a client if you knew what the address:port tuple it was listening on.
* Optimize packet construction functions
Some of the functions that construct packets in
connection.cpp are using a const reference to get the raw
packet data to package and others use a value passed
parameter to do that. The ones that use the value passed
parameter suffer from performance hit as the rather bulky
packet data gets a temporary copy when the parameter is
passed before it lands at its final destination inside the
newly constructed packet. The unnecessary temporary copy
hurts quite badly as the underlying class (SharedBuffer)
actually allocates the space for the data in the heap.
Fix the performance hit by converting all of these value
passed parameters to const references. I believe that this
is what the author of the relevant code actually intended
to do as there is a couple of packet construction helper
functions that already use a const reference to get the
raw data.
* Optimize packet sender thread class
Most of the data sending methods of the packet sender thread
class use a value passed parameter for the packet data to be
sent. This causes the rather bulky data to be allocated on
the heap and copied, slowing the packet sending down. Convert
these parameters to const references to avoid the performance
hit.
* Optimize packet receiver thread class
The packet receiver and processor thread class has many
methods (mostly packet handlers) that receive the packed data
by value. This causes a performance hit that is actually
worse than the one caused by the packet sender methods
because the packet is first handed to the processPacket
method which looks at the packet type stored in the header
and then delegates the actual handling to one of the
handlers. Both, processPacket and all the handlers get the
packet data by value, leading to at least two unnecessary
copies of the data (with malloc and all the slow bells and
whistles of bulky classes).
As there already is a few methods that use a const reference
parameter for the packet data, convert all this value passed
packets to const references.
* Few code updates
* Do not show average RTT before timing out
* Fix unwanted integer division in RTTStatistics
* Fix float format, prettier jitter calculation
* Use +=, 0.1f -> 100.0f for stronger average updates
* Add session_t typedef + remove unused functions
u16 peer_id is used everywhere, to be more consistent and permit some evolutions on this type in the future (i'm working on a PoC), uniformize u16 peer_id to SessionId peer_id
* Move Connection threads to dedicated files + various cleanups
* ConnectionReceiveThread::processPacket now uses function pointer table to route MT packet types
* Various code style fixes
* Code style with clang-format
* Various SharedBuffer copy removal
* SharedBuffer cannot be copied anymore using Buffer
* Fix many SharedBuffer copy (thanks to delete operator)
* Cleanup network headers
* Move peerhandler to a specific header to reduce compilation times
* Move socket.cpp/h to network folder
* More work
* Network code cleanups
* Move socket.{cpp,h} to network folder
* Move Address object to network/address.{cpp,h}
* Move network exceptions to network/networkexceptions.h
* Client: use unique_ptr for Connection
* Server/ClientIface: use shared_ptr for Connection
* Format fixes
* Remove socket.cpp socket.h from clang-format whitelist
* Also fix NetworkPacket code style & make it under clang-format
* Modernize source code: last par
* Use empty when needed
* Use emplace_back instead of push_back when needed
* For range-based loops
* Initializers fixes
* constructors, destructors default
* c++ C stl includes
* Code modernization: subfolders
Modernize various code on subfolders client, network, script, threading, unittests, util
* empty function
* default constructor/destructor
* for range-based loops
* use emplace_back instead of push_back
* C++ STL header style
* Make connection.cpp readable in a pointed place + typo
* Use more for range based loops
* Simplify some tests
* Code style fixes
* connection.h: better PeerChange constructor instead of creating uninitalized object and then affect variables
Fixes a bug where packet reordering made the server give the
client two peer ids instead of one. This in turn confused
reliable packet sending and made connecting to the server fail.
The client usually sends three packets at init: one "dummy"
packet consisting of two 0 bytes, and the init packet as well as
its legacy counterpart. The last one can be turned off since commit
af30183124, but this is of lower
relevance for the bug. The relevant part here is that network
packet reorder (which is a normal occurence) can make the packets
reach the server in different order.
If reorder puts the dummy packet further behind, the following
would happen before the patch:
1. The server will get one of the init packets on channel 1 and
assign the client a peer id, as the packet will have zero as
peer id.
2. The server sends a CONTROLTYPE_SET_PEER_ID packet to inform
the client of the peer id.
3. The next packet from the client will contain the peer id set by
the server.
4. The server sets the m_has_sent_with_id member for the client's
peer structure to true.
5. Now the dummy packet arrives. It has a peer id of zero, therefore
the server searches whether it already has a peer id for the
address the packet was sent from. The search fails because
m_has_sent_with_id was set to true and the server only searched
for peers with m_has_sent_with_id set to false.
6. In a working setup, the server would assign the dummy packet to
the correct peer id. However the server instead now assigns a
second peer id and peer structure to the peer, and assign the
packet to that new peer.
7. In order to inform the peer of its peer id, the server sends a
CONTROLTYPE_SET_PEER_ID command packet, reliably, to the peer.
This packet uses the new peer id.
8. The client sends an ack to that packet, not with the new peer id
but with the peer id sent in 2.
9. This packet reaches the server, but it drops the ACK as the peer
id does not map to any un-ACK-ed packets with that seqnum. The
same time, the server still waits for an ACK with the new peer
id, which of course won't come. This causes the server to
periodically re-try sending that packet, and the client ACKing it
each time.
Steps 7-9 cause annoyances and erroneous output, but don't cause
the connection failure itself.
The actual mistake that causes the connection failure happens in 6:
The server does not assign the dummy packet to the correct peer, but
to a newly created one.
Therefore, all further packets sent by the client on channel 0 are
now buffered by the server as it waits for the dummy packet to reach
the peer, which of course doesn't happen as the server assigned
that packet to the second peer it created for the client.
This makes the connection code indefinitely buffer the
TOSERVER_CLIENT_READY packet, not passing it to higher level code,
which stalls the continuation of the further init process
indefinitely and causes the actual bug.
Maybe this can be caused by reordered init packets as well, the only
studied case was where network has reliably reordered the dummy
packet to get sent after the init packets.
The patch fixes the bug by not ignoring peers where
m_has_sent_with_id has been set anymore. The other changes of the
patch are just cleanups of unused methods and fields and additional
explanatory comments.
One could think of alternate ways to fix the bug:
* The client could simply take the new peer id and continue
communicating with that. This is however worse than the fix as
it requires the peer id set command to be sent reliably (which
currently happens, but it cant be changed anymore). Also, such a
change would require both server and client to be patched in order
for the bug to be fixed, as right now the client ignores peer id
set commands after the peer id is different from
PEER_ID_INEXISTENT and the server requires modification too to
change the peer id internally.
And, most importantly, right now we guarantee higher level server
code that the peer id for a certain peer does not change. This
guarantee would have to be broken, and it would require much
larger changes to the server than this patch means.
* One could stop sending the dummy packet. One may be unsure whether
this is a good idea, as the meaning of the dummy packet is not
known (it might be there for something important), and as it is
possible that the init packets may cause this problem as well
(although it may be possible too that they can't cause this).
Thanks to @auouymous who had originally reported this bug and who
has helped patiently in finding its cause.
* Rename everything.
* Strip J prefix.
* Change UpperCamelCase functions to lowerCamelCase.
* Remove global (!) semaphore count mutex on OSX.
* Remove semaphore count getter (unused, unsafe, depended on internal
API functions on Windows, and used a hack on OSX).
* Add `Atomic<type>`.
* Make `Thread` handle thread names.
* Add support for C++11 multi-threading.
* Combine pthread and win32 sources.
* Remove `ThreadStarted` (unused, unneeded).
* Move some includes from the headers to the sources.
* Move all of `Event` into its header (allows inlining with no new includes).
* Make `Event` use `Semaphore` (except on Windows).
* Move some porting functions into `Thread`.
* Integrate logging with `Thread`.
* Add threading test.
Because we get a Buffer<u8> from ConnectionEvent, don't convert it to SharedBuffer<u8> and return it to Server/Client::Receive which will convert it to NetworkPacket
Instead, put the Buffer<u8> directly to NetworkPacket and return it to packet processing
This remove a long existing memory copy
Also check the packet size directly into Connection::Receive instead of packet processing
NetworkPacket.cpp:
* Remove some deprecated functions, we must use streaming interface
* m_data converted from u8* to std::vector<u8>
* Add an exporter to forge packet to Connection object
* implement operator << std::wstring. n
* implement operator << std::string
* dynamic resize when write packet content.
* fix string writing and performances.
* create ServerCommandFactory, used by client to get useful informations about packet processing (sending).
* Reliability
* Transmit channel
* Implement putRawString for some ugly char (_INIT packet), and use it.
* Many packet read and write migrated
* Implement oldForgePacket to interface writing with current connection
* fix U8/char/bool writing
* fix string writing and performances.
* add some missing functions
* Use v3s16 read instead of reading x,y,z separately
* Add irr::video::SColor support into packets
* Add some missing handlers
* Add a template function to increase offset
* Throw a serialization error on packet reading (must be improved)
PacketFactories:
* Create ServerCommandFactory, used by client to get useful informations about packet processing (sending).
* Create ClientCommandFactory, used by server to get useful informations about packet processing (sending).
Client.cpp:
* implement NetworkPacket ::Send interface.
* Move packet handlers to a dedicated file
* Remove Client::Send(SharedBuffer)
Server.cpp:
* implement NetworkPacket ::Send interface.
* Rewrite all packets using NetworkPacket
* Move packet handlers to a dedicated file
* Remove Server::Send(SharedBuffer)
ClientIface.cpp:
* Remove sendToAll(SharedBuffer<u8>)
Connection.hpp rework:
* Remove duplicate include
* Remove duplicate negation
* Remove a useless variable
* Improve code performance by using a m_peers_list instead of scanning m_peers map
* Remove Connection::Send(SharedBuffer)
* Fix useafterfree into NetworkPacket Sending
* Remove unused Connection::sendToAll
Test.cpp:
* Remove dead code
* Update tests to use NetworkPackets
Misc:
* add new wrappers to Send packets in client, using NetworkPacket
* Add NetworkPacket methods for Connection
* coding style fix
* dead code since changes cleanup
* Use v3s16 read instead of reading x,y,z separately in some packets
* Use different files to handle packets received by client and server
* Cleanup: Remove useless includes
ok @Zeno-
Tested by @Zeno- @VanessaE and @nerzhul on running servers
red-001