Sync to fork (#1)

* [core] Fix crypto mode auto for listener sender (Haivision#2711).


Co-authored-by: oviano <ovcollyer@mac.com>

* [build] Upgraded CI: ubuntu to version 20.04 (Haivision#2682).

* [docs] Added the link for registration in slack to the getting started table (Haivision#2721).

* [core] Fixed FEC Emergency resize crash (Haivision#2717).

Fixed minimum history condition.

* [core] Fixed various compiler warnings on various platforms (Haivision#2679).

* [core] Minor fix of variable shadowing.

* [tests] Minor fix of variable shadowing.

* [build] Add -Wshadow=local to CMake build flags.
Supported since GCC 7.0.

* [core] Correct remaining endianness issues

Fixes the last two remaining test failures on big-endian.  These
operations were all already no-ops on little-endian, and unnecessarily
byteswapped the IP addresses on big-endian.

Closes: Haivision#2697

* [docs] Minor updates to AEAD docs plus changed v1.6.0 to 1.5.2 in some files

* [build] Fix downversioning of _WIN32_WINNT (Haivision#2754).

* [core] Fixed unhandled error in haicrypt (Haivision#2685).

* [core] Use overlapped WSASendTo to avoid loss in UDP sending (Haivision#2632).

* [core] Add volatile keyword to asm block in rdtsc (Haivision#2759).

* [core] Fixed srctime from closing socket was mistakenly cleared

* [core] Fixed group read-ready epoll events.

* [core] Removed unused CUDTGroup::m_Positions.

* [core] Perf improvement of group reading.

* [core] Fixed RCV buffer initialization in Rendezvous.

* [docs] Updating the explicit information for binding to IPv6 wildcard (Haivision#2765).

* [tests] Added custom main with transparent parameters for tests (Haivision#2681).

* [core] Fix memory leak when can't buffer a HS packet (Haivision#2757).

* [core] Refactor CRcvQueue::storePkt(..) for better resource management (Haivision#2775).

* [core] Fix hang up on not enough space in the RCV buffer (Haivision#2745).

When there is space available in the receiving buffer after it is full,
send an ack to allow the sender to resume transmission.
Reschedule sending if ACK decreases the flight span after sending is congested.

Co-authored-by: Maxim Sharabayko <maxlovic@gmail.com>

* [core] fix tsbpd() may deadlock with processCtrlShutdown()

* [core] Slightly optimize the RCV drop by message number (Haivision#2686).

Some minor improvements of logs and comments.

* [core] Rejection not undertaken in rendezvous after KMX failure (Haivision#2692).

* [core] Fix: In rendezvous when processing resulted in ACCEPT it was still sending rejection

* [core] Minor code clean up in CRateEstimator.

* [core] Initialize ISN and PeerISN in CUDT.

* [core] Drop unencrypted packets in GCM mode.

* [apps] Fix the build for target without IP_ADD_SOURCE_MEMBERSHIP (Haivision#2779).

* [core] Added maximum BW limit for retransmissions (Haivision#2714).

* [API] SRT version raised to 1.5.3.

* [apps] Fixed conditional IP_ADD_SOURCE_MEMBERSHIP in testmedia (Haivision#2780).

* [core] Fixed SRT_ATTR_REQUIRES use.

* [build] Added missing public header files in Windows binary installer (Haivision#2784).

The header file access_control.h was added to the source tree
at some point but was not added to the Windows installer.

---------

Co-authored-by: Maxim Sharabayko <maxlovic@gmail.com>
Co-authored-by: oviano <ovcollyer@mac.com>
Co-authored-by: Sektor van Skijlen <ethouris@gmail.com>
Co-authored-by: Maria Sharabayko <41019697+mbakholdina@users.noreply.github.com>
Co-authored-by: Maxim Sharabayko <maxsharabayko@haivision.com>
Co-authored-by: matoro <matoro@users.noreply.github.com>
Co-authored-by: Maria Sharabayko <msharabayko@haivision.com>
Co-authored-by: Steve Lhomme <robux4@ycbcr.xyz>
Co-authored-by: Aaron Jencks <32805004+aaron-jencks@users.noreply.github.com>
Co-authored-by: Guangqing Chen <hi@goushi.me>
Co-authored-by: john <hondaxiao@tencent.com>
Co-authored-by: yomnes0 <127947185+yomnes0@users.noreply.github.com>
Co-authored-by: Mikołaj Małecki <mmalecki@haivision.com>
Co-authored-by: Jose Santiago <jsantiago@haivision.com>
Co-authored-by: Thierry Lelegard <lelegard@users.noreply.github.com>

.github/workflows/android.yaml

@@ -9,7 +9,7 @@ on:
 jobs:
   build:
     name: NDK-R23
-    runs-on: ubuntu-18.04
+    runs-on: ubuntu-20.04
 
     steps:
     - name: Setup Android NDK R23

.github/workflows/cxx11-ubuntu.yaml

@@ -9,7 +9,7 @@ on:
 jobs:
   build:
     name: ubuntu
-    runs-on: ubuntu-18.04
+    runs-on: ubuntu-20.04
 
     steps:
     - uses: actions/checkout@v3

.travis.yml

@@ -71,7 +71,6 @@ matrix:
           - BUILD_TYPE=Release
           - BUILD_OPTS='-DENABLE_MONOTONIC_CLOCK=ON'
 script:
-    - TESTS_IPv6="TestMuxer.IPv4_and_IPv6:TestIPv6.v6_calls_v6*:ReuseAddr.ProtocolVersion:ReuseAddr.*6" ; # Tests to skip due to lack of IPv6 support
     - if [ "$TRAVIS_COMPILER" == "x86_64-w64-mingw32-g++" ]; then
         export CC="x86_64-w64-mingw32-gcc";
         export CXX="x86_64-w64-mingw32-g++";
@@ -95,7 +94,7 @@ script:
       fi
     - if [ "$TRAVIS_COMPILER" != "x86_64-w64-mingw32-g++" ]; then
         ulimit -c unlimited;
-        ./test-srt --gtest_filter="-$TESTS_IPv6";
+        ./test-srt -disable-ipv6;
         SUCCESS=$?;
         if [ -f core ]; then gdb -batch ./test-srt -c core -ex bt -ex "info thread" -ex quit; else echo "NO CORE - NO CRY!"; fi;
         test $SUCCESS == 0;

CMakeLists.txt

@@ -8,7 +8,7 @@
 #
 
 cmake_minimum_required (VERSION 2.8.12 FATAL_ERROR)
-set (SRT_VERSION 1.5.2)
+set (SRT_VERSION 1.5.3)
 
 set (CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/scripts")
 include(haiUtil) # needed for set_version_variables
@@ -153,6 +153,7 @@ option(ENABLE_GETNAMEINFO "In-logs sockaddr-to-string should do rev-dns" OFF)
 option(ENABLE_UNITTESTS "Enable unit tests" OFF)
 option(ENABLE_ENCRYPTION "Enable encryption in SRT" ON)
 option(ENABLE_AEAD_API_PREVIEW "Enable AEAD API preview in SRT" Off)
+option(ENABLE_MAXREXMITBW "Enable SRTO_MAXREXMITBW (v1.6.0 API preview)" Off)
 option(ENABLE_CXX_DEPS "Extra library dependencies in srt.pc for the CXX libraries useful with C language" ON)
 option(USE_STATIC_LIBSTDCXX "Should use static rather than shared libstdc++" OFF)
 option(ENABLE_INET_PTON "Set to OFF to prevent usage of inet_pton when building against modern SDKs while still requiring compatibility with older Windows versions, such as Windows XP, Windows Server 2003 etc." ON)
@@ -299,7 +300,13 @@ if(WIN32)
 	if(ENABLE_INET_PTON)
 		set(CMAKE_REQUIRED_LIBRARIES ws2_32)
 		check_function_exists(inet_pton HAVE_INET_PTON)
-		add_definitions(-D_WIN32_WINNT=0x0600)
+		try_compile(AT_LEAST_VISTA
+			${CMAKE_BINARY_DIR}
+			"${CMAKE_CURRENT_SOURCE_DIR}/scripts/test_vista.c")
+		if(NOT AT_LEAST_VISTA)
+			# force targeting Vista
+			add_definitions(-D_WIN32_WINNT=0x0600)
+		endif()
 	else()
 		add_definitions(-D_WIN32_WINNT=0x0501)
 	endif()
@@ -460,6 +467,13 @@ if (USE_GNUSTL)
 	set (SRT_LIBS_PRIVATE ${SRT_LIBS_PRIVATE} ${GNUSTL_LIBRARIES} ${GNUSTL_LDFLAGS})
 endif()
 
+if (ENABLE_MAXREXMITBW)
+	add_definitions(-DENABLE_MAXREXMITBW)
+	message(STATUS "MAXREXMITBW API: ENABLED")
+else()
+	message(STATUS "MAXREXMITBW API: DISABLED")
+endif()
+
 if (USING_DEFAULT_COMPILER_PREFIX)
 # Detect if the compiler is GNU compatible for flags
 if (CMAKE_CXX_COMPILER_ID MATCHES "GNU|Intel|Clang|AppleClang")
@@ -642,6 +656,9 @@ endif()
 # add extra warning flags for gccish compilers
 if (HAVE_COMPILER_GNU_COMPAT)
 	set (SRT_GCC_WARN "-Wall -Wextra")
+	if (CMAKE_CXX_COMPILER_VERSION VERSION_GREATER 7.0 AND CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
+		set (SRT_GCC_WARN "${SRT_GCC_WARN} -Wshadow=local")
+	endif()
 else()
 	# cpp debugging on Windows :D
 	#set (SRT_GCC_WARN "/showIncludes")

README.md

@@ -144,14 +144,14 @@ In live streaming configurations, the SRT protocol maintains a constant end-to-e
 
 ## Getting Started with SRT
 
-|                                                                                                                               |                                                                                    |                                                                                   |
-|:-----------------------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------:|
-| [The SRT API](./docs#srt-api-documents)                                                                                       | [IETF Internet Draft](https://datatracker.ietf.org/doc/html/draft-sharabayko-srt-01)    | [Sample Apps](./docs#sample-applications)                                         |
-| Reference documentation for the SRT library API                                                                               | The SRT Protocol Internet Draft                                                    | Instructions for using test apps (`srt-live-transmit`, `srt-file-transmit`, etc.) |
-| [SRT Technical Overview](https://github.com/Haivision/srt/files/2489142/SRT_Protocol_TechnicalOverview_DRAFT_2018-10-17.pdf)  | [SRT Deployment Guide](https://www.srtalliance.org/srt-deployment-guide/)          | [SRT CookBook](https://srtlab.github.io/srt-cookbook)                             |
-| Early draft technical overview (precursor to the Internet Draft)                                                          | A comprehensive overview of the protocol with deployment guidelines                | Development notes on the SRT protocol                                             |
-| [Innovation Labs Blog](https://medium.com/innovation-labs-blog/tagged/secure-reliable-transport)                              | [SRTLab YouTube Channel](https://www.youtube.com/channel/UCr35JJ32jKKWIYymR1PTdpA) | [Slack](https://srtalliance.slack.com)                                            |
-| The blog on Medium with SRT-related technical articles                                                                        | Technical YouTube channel with useful videos                                       | Slack channels to get the latest updates and ask questions                        |
+|                                                                                                                               |                                                                                      |                                                                                   |
+|:-----------------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------:|
+| [The SRT API](./docs#srt-api-documents)                                                                                       | [IETF Internet Draft](https://datatracker.ietf.org/doc/html/draft-sharabayko-srt-01) | [Sample Apps](./docs#sample-applications)                                         |
+| Reference documentation for the SRT library API                                                                               | The SRT Protocol Internet Draft                                                      | Instructions for using test apps (`srt-live-transmit`, `srt-file-transmit`, etc.) |
+| [SRT Technical Overview](https://github.com/Haivision/srt/files/2489142/SRT_Protocol_TechnicalOverview_DRAFT_2018-10-17.pdf)  | [SRT Deployment Guide](https://www.srtalliance.org/srt-deployment-guide/)            | [SRT CookBook](https://srtlab.github.io/srt-cookbook)                             |
+| Early draft technical overview (precursor to the Internet Draft)                                                              | A comprehensive overview of the protocol with deployment guidelines                  | Development notes on the SRT protocol                                             |
+| [Innovation Labs Blog](https://medium.com/innovation-labs-blog/tagged/secure-reliable-transport)                              | [SRTLab YouTube Channel](https://www.youtube.com/channel/UCr35JJ32jKKWIYymR1PTdpA)   | [Slack](https://srtalliance.slack.com)                                            |
+| The blog on Medium with SRT-related technical articles                                                                        | Technical YouTube channel with useful videos                                         | Slack channels to get the latest updates and ask questions <br />[Join SRT Alliance on Slack](https://slackin-srtalliance.azurewebsites.net/) |
 
 ### Additional Documentation
 

apps/apputil.cpp

@@ -49,9 +49,12 @@ int inet_pton(int af, const char * src, void * dst)
    ZeroMemory(&ss, sizeof(ss));
 
    // work around non-const API
-   strncpy(srcCopy, src, INET6_ADDRSTRLEN + 1);
+#ifdef _MSC_VER
+   strncpy_s(srcCopy, INET6_ADDRSTRLEN + 1, src, _TRUNCATE);
+#else
+   strncpy(srcCopy, src, INET6_ADDRSTRLEN);
    srcCopy[INET6_ADDRSTRLEN] = '\0';
-
+#endif
    if (WSAStringToAddress(
       srcCopy, af, NULL, (struct sockaddr *)&ss, &ssSize) != 0)
    {

apps/socketoptions.hpp

@@ -258,6 +258,9 @@ const SocketOption srt_options [] {
 #ifdef ENABLE_AEAD_API_PREVIEW
     ,{ "cryptomode", 0, SRTO_CRYPTOMODE, SocketOption::PRE, SocketOption::INT, nullptr }
 #endif
+#ifdef ENABLE_MAXREXMITBW
+    ,{ "maxrexmitbw", 0, SRTO_MAXREXMITBW, SocketOption::POST, SocketOption::INT64, nullptr }
+#endif
 };
 }
 

apps/srt-live-transmit.cpp

@@ -852,8 +852,14 @@ int main(int argc, char** argv)
 void TestLogHandler(void* opaque, int level, const char* file, int line, const char* area, const char* message)
 {
     char prefix[100] = "";
-    if ( opaque )
-        strncpy(prefix, (char*)opaque, 99);
+    if ( opaque ) {
+#ifdef _MSC_VER
+        strncpy_s(prefix, sizeof(prefix), (char*)opaque, _TRUNCATE);
+#else
+        strncpy(prefix, (char*)opaque, sizeof(prefix) - 1);
+        prefix[sizeof(prefix) - 1] = '\0';
+#endif
+    }
     time_t now;
     time(&now);
     char buf[1024];

apps/srt-tunnel.cpp

@@ -299,7 +299,7 @@ class Tunnel
 
     Tunnel(Tunnelbox* m, std::unique_ptr<Medium>&& acp, std::unique_ptr<Medium>&& clr):
         parent_box(m),
-        med_acp(move(acp)), med_clr(move(clr)),
+        med_acp(std::move(acp)), med_clr(std::move(clr)),
         acp_to_clr(this, med_acp.get(), med_clr.get(), med_acp->id() + ">" + med_clr->id()),
         clr_to_acp(this, med_clr.get(), med_acp.get(), med_clr->id() + ">" + med_acp->id())
     {
@@ -649,7 +649,7 @@ void TcpMedium::CreateListener()
 
     sockaddr_any sa = CreateAddr(m_uri.host(), m_uri.portno());
 
-    m_socket = socket(sa.get()->sa_family, SOCK_STREAM, IPPROTO_TCP);
+    m_socket = (int)socket(sa.get()->sa_family, SOCK_STREAM, IPPROTO_TCP);
     ConfigurePre();
 
     int stat = ::bind(m_socket, sa.get(), sa.size());
@@ -694,7 +694,7 @@ unique_ptr<Medium> SrtMedium::Accept()
 unique_ptr<Medium> TcpMedium::Accept()
 {
     sockaddr_any sa;
-    int s = ::accept(m_socket, (sa.get()), (&sa.syslen()));
+    int s = (int)::accept(m_socket, (sa.get()), (&sa.syslen()));
     if (s == -1)
     {
         Error(errno, "accept");
@@ -726,7 +726,7 @@ void SrtMedium::CreateCaller()
 
 void TcpMedium::CreateCaller()
 {
-    m_socket = ::socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
+    m_socket = (int)::socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
     ConfigurePre();
 }
 
@@ -850,7 +850,7 @@ Medium::ReadStatus Medium::Read(bytevector& w_output)
     size_t pred_size = shift + m_chunk;
 
     w_output.resize(pred_size);
-    int st = ReadInternal((w_output.data() + shift), m_chunk);
+    int st = ReadInternal((w_output.data() + shift), (int)m_chunk);
     if (st == -1)
     {
         if (IsErrorAgain())
@@ -884,7 +884,7 @@ Medium::ReadStatus Medium::Read(bytevector& w_output)
 
 void SrtMedium::Write(bytevector& w_buffer)
 {
-    int st = srt_send(m_socket, w_buffer.data(), w_buffer.size());
+    int st = srt_send(m_socket, w_buffer.data(), (int)w_buffer.size());
     if (st == SRT_ERROR)
     {
         Error(UDT::getlasterror(), "srt_send");
@@ -907,7 +907,7 @@ void SrtMedium::Write(bytevector& w_buffer)
 
 void TcpMedium::Write(bytevector& w_buffer)
 {
-    int st = ::send(m_socket, w_buffer.data(), w_buffer.size(), DEF_SEND_FLAG);
+    int st = ::send(m_socket, w_buffer.data(), (int)w_buffer.size(), DEF_SEND_FLAG);
     if (st == -1)
     {
         Error(errno, "send");
@@ -971,7 +971,7 @@ struct Tunnelbox
         lock_guard<std::mutex> lk(access);
         Verb() << "Tunnelbox: Starting tunnel: " << acp->uri() << " <-> " << clr->uri();
 
-        tunnels.emplace_back(new Tunnel(this, move(acp), move(clr)));
+        tunnels.emplace_back(new Tunnel(this, std::move(acp), std::move(clr)));
         // Note: after this instruction, acp and clr are no longer valid!
         auto& it = tunnels.back();
 
@@ -1191,7 +1191,7 @@ int main( int argc, char** argv )
             Verb() << "Connected. Establishing pipe.";
 
             // No exception, we are free to pass :)
-            g_tunnels.install(move(accepted), move(caller));
+            g_tunnels.install(std::move(accepted), std::move(caller));
         }
         catch (...)
         {

apps/transmitmedia.cpp

@@ -805,7 +805,7 @@ class UdpCommon
 
     void Setup(string host, int port, map<string,string> attr)
     {
-        m_sock = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
+        m_sock = (int)socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
         if (m_sock == -1)
             Error(SysError(), "UdpCommon::Setup: socket");
 
@@ -848,7 +848,6 @@ class UdpCommon
 
         if (is_multicast)
         {
-            ip_mreq_source mreq_ssm;
             ip_mreq mreq;
             sockaddr_any maddr (AF_INET);
             int opt_name;
@@ -872,6 +871,7 @@ class UdpCommon
             if (attr.count("source"))
             {
 #ifdef IP_ADD_SOURCE_MEMBERSHIP
+                ip_mreq_source mreq_ssm;
                 /* this is an ssm.  we need to use the right struct and opt */
                 opt_name = IP_ADD_SOURCE_MEMBERSHIP;
                 mreq_ssm.imr_multiaddr.s_addr = sadr.sin.sin_addr.s_addr;
@@ -1144,7 +1144,7 @@ extern unique_ptr<Base> CreateMedium(const string& uri)
     }
 
     if (ptr.get())
-        ptr->uri = move(u);
+        ptr->uri = std::move(u);
 
     return ptr;
 }

apps/uriparser.cpp

@@ -156,7 +156,7 @@ void UriParser::Parse(const string& strUrl, DefaultExpect exp)
     if (idx != string::npos)
     {
         m_host   = strUrl.substr(0, idx);
-        iQueryStart = idx + 1;
+        iQueryStart = (int)(idx + 1);
     }
     else
     {
@@ -297,12 +297,12 @@ void UriParser::Parse(const string& strUrl, DefaultExpect exp)
         if (idx != string::npos)
         {
             strQueryPair = strUrl.substr(iQueryStart, idx - iQueryStart);
-            iQueryStart = idx + 1;
+            iQueryStart = (int)(idx + 1);
         }
         else
         {
             strQueryPair = strUrl.substr(iQueryStart, strUrl.size() - iQueryStart);
-            iQueryStart = idx;
+            iQueryStart = (int)idx;
         }
 
         idx = strQueryPair.find("=");

docs/API/API-functions.md

@@ -171,7 +171,7 @@ Since SRT v1.5.0.
 | [SRT_REJ_FILTER](#SRT_REJ_FILTER)            | 1.3.4     | The [`SRTO_PACKETFILTER`](API-socket-options.md#SRTO_PACKETFILTER) option has been set differently on both connection parties  |
 | [SRT_REJ_GROUP](#SRT_REJ_GROUP)              | 1.4.2     | The group type or some group settings are incompatible for both connection parties                             |
 | [SRT_REJ_TIMEOUT](#SRT_REJ_TIMEOUT)          | 1.4.2     | The connection wasn't rejected, but it timed out                                                               |
-| [SRT_REJ_CRYPTO](#SRT_REJ_CRYPTO)            | 1.6.0-dev | The connection was rejected due to an unsupported or mismatching encryption mode                               |
+| [SRT_REJ_CRYPTO](#SRT_REJ_CRYPTO)            | 1.5.2     | The connection was rejected due to an unsupported or mismatching encryption mode                               |
 | <img width=290px height=1px/>                |           |                                                                                                                |
 
 <h4 id="error-codes">Error Codes</h4>
@@ -359,8 +359,12 @@ int srt_bind(SRTSOCKET u, const struct sockaddr* name, int namelen);
 Binds a socket to a local address and port. Binding specifies the local network
 interface and the UDP port number to be used for the socket. When the local 
 address is a wildcard (`INADDR_ANY` for IPv4 or `in6addr_any` for IPv6), then
-it's bound to all interfaces (although see `SRTO_IPV6ONLY` and additional
-information below for details about the wildcard address in IPv6).
+it's bound to all interfaces.
+
+**IMPORTANT**: When you bind an IPv6 wildcard address, note that the
+`SRTO_IPV6ONLY` option must be set on the socket explicitly to 1 or 0 prior to
+calling this function. See
+[`SRTO_IPV6ONLY`](API-socket-options.md#SRTO_IPV6ONLY) for more details.
 
 Binding is necessary for every socket to be used for communication. If the socket
 is to be used to initiate a connection to a listener socket, which can be done,