======================================================================= FNET CURRENT (2006-04-24) ....................................................................... Misc ---- - (2007-01-12) Make enabling/disabling the direct write optimization runtime rather than compile time. Bugfixes -------- - (2007-01-09) Don't open new channels unless the channel id was invented by our peer. Also, don't send a reply when an rpc request error occurs. (avoid infinite error response loop) - (2006-09-21) Handle BAD_REQUEST/NULL method name case. When a bad request was handled, the method name (NULL) would cause the method lookup to crash. This should be resolved now. - On connection close: discard packets from the inner packet queue even if the outer packet queue is empty. Not doing this could trigger a resource leak if a dummy packet responsible for cleanup (and channel closing) was left in the inner queue after the connection was closed. ======================================================================= FNET V_2_0_0_RELEASE ....................................................................... New Dependencies ---------------- - vespalib (for test framework) Removed Code/Features --------------------- - Moved the core test framework into vespalib. FNET now depends on vespalib for this. The FNET test framework is now implemented on top of the test framework supplied by vespalib. Also, the vespalib test script (testrun.sh) is used to control the running of FNET tests. - Removed FNET_LogHandler class. This removes the simple logging framework located in FNET. It was deprecated several versions ago, and all internal logging in FNET is now done using the new log module. - Removed OO RPC Support (RPC with support for objects and classes). This is done to simplify the overall design of RPC, which also makes it simpler to implement compatible components in other languages. RPC packet codes and error codes have been re-enumerated to cover up the holes left by the removed support for OO RPC. The protocol documentation has been modified to reflect the new RPC design. NB: this makes RPC incompatible with older versions of FNET. Spiral ------ Spiral is a simplified single-threaded RPC abstraction layer. It is intended for truly single-threaded applications that does not want to mess around with the complete FNET API. Spiral currently only supports the RPC client aspect. New Features ------------ - Added support for limiting the size of input/output buffers for connections. This is done by invoking the FNET_Transport::SetMaxInputBufferSize and FNET_Transport::SetMaxOutputBufferSize methods. - Added FNET_Connection::GetQueueLen method that may be used to obtain the number of packets currently located in the packet output queue. - Added support for enable/disable read and disable write events for io components. This is used by the support to select on external file descriptors. - Added support to shrink databuffers. This is used by the support to limit input/output buffers. - Added support for selecting on external file descriptors by using the FNET IO component framework. This includes using the FNET_FDSelector class and the FNET_IFDSelectorHandler interface. - Better support for single-threaded compile. The test system has been modified to work when compiled single-threaded. A new test macro named TEST_THREADS may be used by tests that need threads to run. This will result in the test begin skipped if threads are not supported. - Added FRT_ISharedBlob interface with corresponding support. This may be used to share the same memory regions between multiple requests. Also, all shared blobs will be subref'ed by the request as soon as the request packet is encoded to the connection output buffer. Bugfixes -------- - Fixed a minor bug in the xsync daemon; close other end of pipe after fork, to avoid hang in mother process if child fails. ======================================================================= FNET V_1_4_9_RELEASE ....................................................................... Robustness ---------- - Make task scheduling more robust in cases where the event loop latency is very high. (ref Bug #462022) ======================================================================= FNET V_1_4_8_RELEASE ....................................................................... ======================================================================= FNET V_1_4_7_RELEASE ....................................................................... Misc ---- - Minor timing adjustments in the RPC invoke test to make it work better. ======================================================================= FNET V_1_4_6_RELEASE ....................................................................... ======================================================================= FNET V_1_4_5_RELEASE ....................................................................... ======================================================================= FNET V_1_4_4_RELEASE ....................................................................... Misc ---- - avoid some warnings with gcc 4 (weffc++) Bugfixes -------- - check if _adminChannel is closed while waiting for callbacks (in the CloseAdminChannel method) ======================================================================= FNET V_1_4_3_RELEASE ....................................................................... Misc ---- - use vtag stuff in fastos to report fnet version (to avoid empty release hassle) - Better support for make -j ======================================================================= FNET V_1_4_2_RELEASE ....................................................................... [NB: no changes at all; still identifies as 1.4.1] ======================================================================= FNET V_1_4_1_RELEASE ....................................................................... ======================================================================= FNET V_1_4_0_RELEASE ....................................................................... Public testing toolkit ---------------------- The FNET testing toolkit has been made public and is now a part of the fnet library. The macros have been renamed to all start with 'TEST_'. The local fnet tests have been modified and an example test program has been added. New logging API --------------- All logging has been converted to use the new "log" module and its API. All functions in class FNET_LogHandler are now deprecated, in particular FNET_LogHandler::SetLogMask and FNET_LogHandler::SetLogHandler no longer have any effect. All users of fnet will need to link with the "log" library. Misc ---- - thread_id test is void'ed if fnet is compiled without threads. ======================================================================= FNET V_1_3_0_RELEASE ....................................................................... FRT/RPC Lobotomy/Cleanup ------------------------ The support for local invocation (and location transparency) is removed. The code is also somewhat simplified to reflect the now less general nature of things. It hurts to see so much thought go down the drain, but it greatly reduces maintenance complexity and simplifies future development. Who needs sub microsecond invocations anyway... Changed code to avoid warning messages produced by gcc 3.3 and later when compiling with the -Weffc++ option. Removed Code ------------ - FNET_ITransportHook - FNET_IPingable - FNET_Pinger - FNET_ServerInfo - FNET_Transport::SetMinEventTimeOut - FNET_Transport::SetPingInterval - FNET_Transport::AddPingTarget - FNET_Transport::RemovePingTarget - FNET_Transport::HookNow New Features ------------ - Added preliminary support for 2-way RPC. - Added support for invoking RPC methods without having an FRT_Target. - Added a Reset() method to the reflection manager that removes all methods. - Added a method mismatch rpc hook that may be used to catch method calls not matching any methods. - Added support for rpc session state (All rpc calls made on a single connection is referred to as a session). This was done by adding a context to each connection (used to store the session state), making it possible to obtain the underlying connection given the rpc request object and adding special rpc hooks that are called when a connection is established, when a connection is lost and right before a connection is deleted. - The transport object now comes with a built-in scheduler object. - The FNET_Scheduler constructor may be given the current time. This simplifies running the scheduler in a simulated environment. - A Sync method has been added to the connection class. This method may be used to block until all previously posted packets have been encoded. - Added InTub method to the memorytub class. This method may be used to check if the tub owns a given byte of memory. This feature is primarily used for regression testing. - The FNET_Connector class has been given a GetPortNumber method to be able to extract the port number of the underlying server socket. - Allow listening on port number 0. - Add FRT_Target::GetConnection() method. Only returns a pointer to the connection owned by the target. No new references are allocated. API changes ----------- - The FRT supervisor may be constructed either in stand-alone mode, or based on an external transport object and an external thread pool. The transport proxy methods may only be used when in stand-alone mode. - The scheduler no longer implements the transport hook interface (removed). The virtual Tick method has been replaced by the non-virtual CheckTasks method. - The transport Main/Start methods no longer take a transport hook as parameter (the interface has been removed). Similar functionality may now be obtained by using the transport InitEventLoop and EventLoopIteration methods. - The RPC target InvokeAsync method now returns bool, true means that the method completed immediately. false means the invocation is in progress. - The FRT_IRequestWait::RequestDone method now takes a second parameter; a bool indicating whether this invocation is done synchronously (directly from the invoke method). Internal changes ---------------- - Added IOC CleanupHook method that is called right before an IOC is deleted. This is used by the FNET_Connection class to support a connection cleanup callback which in turn is used by the FRT_Supervisor to activate the SessionFini hook needed for RPC session support. - Statistics updating is now done with a scheduled stats update task. This reduces the complexity of the event loop. - The connection write method now avoids writing 0 bytes to the network. Also, empty writes are silently ignored since using the packet queue for control packets is perfectly legal. - Channel IDs are now chosen in a way that is future compatible with 2-way channel creation. Misc ---- - Store listen/connect spec in IOC and use it in IOC related log messages. - Added rpc proxy example program that may be useful for debugging. - Added FNET_DummyPacket helper packet class that may be used to implement hooks in the connection output packet queue. - The FNET_Packet::NoPCODE() and FNET_Connection::NoID() methods have been replaced by the FNET_NOID macro. - Added FNET_ prefix to DISABLE_DIRECT_WRITE compile flag. - Added FNET_SANITY_CHECKS compile flag. This enables some extra sanity checks in the event loop. - Added generic cross-host testing support as part of testing framework (xtest make target removed). This includes the xsync daemon used to sync test states between hosts. The environment variables PORTBASE and PEER are used to control testing behavior. - In regression test framework: enable output from passed test cases to be collapsed to reduce logging. - Added support for mutex instrumentation. This requires a version of FastOS supporting instrumented mutexes. - Added FRT_Supervisor::GetListenPort() method. Bugfixes -------- - RPC: Handle request return (after request detach) while still inside sync invocation method correctly (also added to regress test). - RPC: ensure that the cleanup handler is run also when the request is recycled by the user without making the trip back to the request pool. ======================================================================= FNET V_1_2_4_RELEASE ....................................................................... Bugfixes -------- - Wait for direct write to complete before discarding packets when the connection is going down. ======================================================================= FNET V_1_2_3_RELEASE ....................................................................... New Features ------------ - Added support for sending multiple packets at the same time. This may reduce TCP latency when used with Nagle's algorithm. ======================================================================= FNET V_1_2_2_RELEASE ....................................................................... Bugfixes -------- - Fix Makefile to work with parallel compilation (gcc -j option). ======================================================================= FNET V_1_2_1_RELEASE ....................................................................... Misc ---- - Max events per event loop iteration: 256 -> 4096 Bugfixes -------- - Avoid overlap between chunk header and chunk data in FRT memorytub. - Remove race around single req wait where condition could be destructed while being locked. - Don't set connection state to closed if connect fails synchronously. This caused a channel lost event to be sent on the admin channel that caused a deadlock in the serverinfo class. It is also an API improvement, since connect now fails either synchronously or asynchronously (not both). ======================================================================= FNET V_1_2_0_RELEASE ....................................................................... New Features ------------ - Added support to construct an FNET_DataBuffer based on a preallocated buffer. - Added support for scheduling a task to be run as soon as possible. - Added support for killing a task so that it may not be scheduled in the future. - Added databuffer methods to write data without checking for free space. These methods have a 'Fast' suffix. - Added AssertValid method that may be called anytime to assert that the databuffer is in a legal state. - Added endian detection. Use the FNET_Info::GetEndian method to obtain host endian. - Added methods to read/peek information stored in reverse internet order (little endian) in a databuffer. - VPATH compilation. Internal Changes ---------------- - Removed asserts used to check for valid data in databuffer read methods. - When posting a packet, try to write it to the network directly (if there is no other pending write operations). Post an enable write event to the transport thread if there is pending work after a direct write has been attempted. This behavior may be disabled by using the --disable-direct-write configure option. Misc ---- - Added static instance of FNET_CMD_NOCOMMAND control packet for completeness. - Moved fastserver stuff to the fastserver4 cvs module. - Simple regress test framework. Run tests with 'make test' from fnet/src. Use 'make xtest' to run cross-host testing. This requires setting the PORT and PEER environment variables to sane value. Example: to run a cross-host test between host A and host B using port 8000 (on both hosts); run 'make PORT=8000 PEER=B xtest' on host A and 'make PORT=8000 PEER=A xtest' on host B. Bugfixes -------- - Set connection state correctly if socket connect completes synchronously. FRT ([F]NET [R]emote [T]ools) [middle-ware library] --------------------------------------------------- The FRT library is a layer on top of FNET supporting a proprietary RPC protocol. In addition to the network protocol, the FRT library also contains API support for location transparent dynamic method invocation (examples and regress tests also included). ======================================================================= FNET V_1_1_2_RELEASE ....................................................................... Bugfixes -------- - Handle EOL returned from socket read. This lets FNET handle peers that close connections nicely. Previously this would result in a very tight event loop reading EOL from the socket in each iteration. - Avoid time measurement errors in scheduler by using absolute values for bucket timeouts rather than accumulated delta values. If the scheduler was invoked very often, it would appear that time stood still. ======================================================================= FNET V_1_1_1_RELEASE ....................................................................... Compiler nitpicks ----------------- - Renamed TICK to SLOT_TICK in scheduler class to avoid conflict with macro on Solaris. - Removed comma at end of enums in fs4 example packet library. - Use public inheritance in fsearch_maxthreads fs4 example application. - Conceal infinite loop in fsearch_somethreads fs4 example application. - Cast error-code to signed value before applying negation in fs4 searchtest example application. - Cast enum to uint32_t to avoid constructor confusion in proxy example application. - Explicitly cast stat values to float. Misc ---- - Removed memory leak in scheduling test to avoid HUGE memory leak report on application exit. ======================================================================= FNET V_1_1_0_RELEASE ....................................................................... API Changes ----------- Added method bool FNET_IServerAdapter::InitAdminChannel(FNET_Channel *) That must be implemented by server applications using FNET. The method is called when a new incoming connection has been established in order to let the application keep track of its current clients. This channel works just like normal channels. The only difference is that it is opened before any packets are received, thereby allowing the application to implement connection-oriented stuff like proxies. Implementing this method to always return false will preserve old behavior. NOTE: closing the connection directly (by invoking Close()) is allowed from the InitAdminChannel method (This is an exception from the general rule that the application should close all connections with async events. The reason for this exception is to allow better resource management). Added an FNET_Context parameter to the FNET_Connection constructor and to the FNET_Transport::Connect method. This was done to enable the application to set the admin channel context in the client end-point of a connection. Also, VOIDP=NULL is used as default context, instead of the previously used INT=0xffffffff (FNET_Connection::NoID()). Added a parameter of type FNET_ITransportHook to the FNET_Transport::Start method in order to enable sharing the transport thread without the need to create a wrapper object (The parameter is optional with NULL as default value). Made the FNET_ITransportHook parameter to the FNET_Transport::Main method optional (default value is NULL). Server info objects may now be marked as 'bad' [MarkBad()/ClearBad()/IsBad()]. If a server (info) is marked as bad, FNET will try to keep it down, rather than up. Note that node badness may be set by the application code in any thread and is not protected by any locks. The most sensible use of this flag would be to use it as an async signal, raising it if a server goes bad, and never lowering it again (ClearBad() is implemented for completeness, but needs to be used with care). Added new method FNET_Channel::CloseAndFree() that may be used to close and free a channel in a single operation. This is equivalent with first invoking Close(), then Free(), but is more efficient. The packet streamer interface has been modified in order to support context-dependent un-streaming of packets and variable length packet headers. This has resulted in the following method changes: ----- bool FNET_IServerAdapter::InitChannel(FNET_Channel *channel, FNET_Packet *packet) ----- || \/ ----- bool FNET_IServerAdapter::InitChannel(FNET_Channel *channel, uint32_t pcode) ----- This reflects the fact that in order to do context-dependent packet un-streaming, we need to resolve the context of the channel that should receive the packet before un-streaming the packet. This also means that the packet itself is not available when opening server-side channels. ----- FNET_Packet * FNET_IPacketFactory::CreatePacket(uint32_t pcode) ----- || \/ ----- FNET_Packet * FNET_IPacketFactory::CreatePacket(uint32_t pcode, FNET_Context context) ----- The context is propagated to the packet factory in order to eliminate the need to change the FNET_Packet::Decode(...) method. ----- uint32_t FNET_IPacketStreamer::GetHeaderLen() uint32_t FNET_IPacketStreamer::GetPacketLen(FNET_DataBuffer *src) ----- || \/ ----- bool FNET_IPacketStreamer::GetPacketInfo(FNET_DataBuffer *src, uint32_t *plen, uint32_t *pcode, uint32_t *chid, bool *broken) ----- The packet info is extracted before the packet is un-streamed in order to locate the channel that should receive the packet. This change gives us support for variable length packet headers and a way to signal that the connection is broken. ----- FNET_Packet * FNET_IPacketStreamer::Decode(FNET_DataBuffer *src, uint32_t *chid) ----- || \/ ----- FNET_Packet * FNET_IPacketStreamer::Decode(FNET_DataBuffer *src, uint32_t plen, uint32_t pcode, FNET_Context context) ----- The packet info is injected as parameters to avoid having to extract it from the packet a second time. Note that the chid has been replaced by the context of the channel that will receive the packet. This completes the support for context-dependent packet un-streaming. Internal Changes ---------------- Allow IOC_DELETE event before IOC_ADD event. This enables the application to post an event indicating that the connection itself should be closed when invoked to open the admin channel on that connection (only applies to server connections). Optimized FNET_DataBuffer by using pointers to define dead/data/free sections of the buffer. Each connection now has its own FNET_ChannelPool object instead of sharing an FNET_ObjectPool. This reduces locking. The methods on FNET_Channel has been un-inlined in order to instantiate FNET_Connection objects with an inlined FNET_ChannelPool object which in turn has inline FNET_Channel objects. Make sure that the packets posted on a connection are freed in the same order as they were posted, even if the connection goes down. This is done to enable the application to use packet free order for syncing purposes. It also enables the usage of packets that perform code in the free method that requires that all previously posted packets have already been freed (Like memory pool cleanup code). Changed some log messages to FNET_LOG_DEBUG. New Control Packets ------------------- - FNET_CMD_TIMEOUT : This control packet command may be used to signal a timeout. Note that FNET does not use timeout packets internally. They are only included to ease the implementation of timeout signaling in applications using FNET. - FNET_CMD_BAD_PACKET : This control packet signals the reception of a packet that could not be un-streamed. It will be delivered in place of the packet that was invalid. New Classes ----------- - FNET_Task (schedulable tasks; timeouts etc.) - FNET_Scheduler (class for scheduling tasks) - FNET_ChannelPool (per-connection channel pooling) Removed Classes --------------- - FNET_ObjectPool New Examples ------------ - proxy : A simple proxy implementation showing how FNET may be used to handle raw data as packets and also how to focus on connections rather than channels. - timeout : Simple application showing how to use the task/scheduler classes to implement timeout support. Bugfixes -------- - use maxreconnectdelay if the connect failed synchronously. - don't try to connect to target host if basic socket init fails. ======================================================================= FNET V_1_0_0_RELEASE ....................................................................... Initial release. NOTE: 1.0.x is no longer supported.