various fixes

CHANGELOG.md

@@ -1,8 +1,9 @@
 ## XX.XX.XX
 - ! Minor breaking change ! The "Consent required" option of the SDK Behavior Settings now applies. While it is on, the SDK collects nothing until the app gives consent; while it is off, the SDK works as if every consent were given, also in apps that require consent.
+- ! Minor breaking change ! The "Automatic session tracking" option of the SDK Behavior Settings now applies. While it is off, the SDK tracks no session by itself and the app's session calls work, as with `enableManualSessionControl()`; while it is on, the SDK tracks sessions also in apps that called `enableManualSessionControl()`. A change while the SDK runs ends or begins the session.
 - ! Minor breaking change ! Every request is now sent as POST, with its data in the request body. A client set with `setHTTPClient` gets `usePost` true for every request and must support POST. `alwaysUsePost` is deprecated and has no effect.
-- ! Minor breaking change ! `start()` now refuses a server URL without `http://` or `https://` or without a host, an empty app key and the placeholder `YOUR_APP_KEY`: it logs an error and the SDK stays off. Before, a URL without a scheme was used over plain HTTP.
-- ! Minor breaking change ! Recorded events are now sent on the session update interval (60 seconds by default) or when the event queue threshold is reached, instead of every 3 seconds. `flushEvents()` moves them into a request right away.
+- ! Minor breaking change ! `start()` now refuses a server URL without `http://` or `https://`, without a host, with a port that is not a number from 1 to 65535, or with a query or a fragment, an empty app key and the placeholder `YOUR_APP_KEY`: it logs an error and the SDK stays off. Before, a URL without a scheme was used over plain HTTP.
+- ! Minor breaking change ! Recorded events are now sent on the session update interval (60 seconds by default) or when the event queue threshold is reached, instead of every 3 seconds. `flushEvents()` moves them into a request right away. Feedback answers (`[CLY]_nps`, `[CLY]_survey`, `[CLY]_star_rating`) and `[CLY]_push_action` events are moved into a request as soon as they are recorded.
 - ! Minor breaking change ! `beginSession`, `updateSession` and `endSession` now work only with `enableManualSessionControl()`; with automatic sessions they are ignored with a warning. A manual `updateSession()` now always reports the session duration, also when called sooner than the session update interval.
 - ! Minor breaking change ! Remote config downloads now use the current remote config request and no longer enroll the user in A/B tests; call `enrollABOnRCDownload()` before `start()` to keep enrolling. With `enableRemoteConfig()`, values are downloaded at the end of `start()`, after a device ID change and when the consent is given, instead of at every session start.
 - ! Minor breaking change ! Config values below their minimum are now ignored with a warning instead of being raised: 0 for the internal limits, `setMaxRequestQueueSize` and `setAutomaticSessionUpdateInterval`, and values below 1 for `setEventsToRQThreshold`. `setEventsToRQThreshold` also keeps values above 10000; the event queue still sends at 1000 events.
@@ -23,14 +24,14 @@
 - Added `disableHealthCheck()` for turning off the health check request.
 - Added `status_code` and `body` to `HTTPResponse`. A custom HTTP client can fill them so that the health check reports the last request the server refused; a client that leaves them empty works as before.
 - Calls made before `start()`, a second `start()`, beginning a session that already runs and ending one that does not are now logged at warning level. Failed and non-JSON server answers, and session calls ignored because the SDK Behavior Settings turned session tracking off, are now logged at debug level.
-- Added device ID generation: when no device ID is stored and the app gives none before `start()`, the SDK generates one. `getDeviceID()` and `getDeviceIDType()` return the device ID in use and where it came from. Every request carries the device ID type (`t`).
+- Added device ID generation: when no device ID is stored and the app gives none before `start()`, the SDK generates one. `getDeviceID()` and `getDeviceIDType()` return the device ID in use and where it came from. Every request carries the device ID type (`t`). After `start()`, `setDeviceID(id)` merges a generated device ID into the new one, as for a user who logs in, and changes a device ID the app gave without merge; `setDeviceID(id, same_user)` still decides explicitly.
 - Added `enableClearStoredDeviceID()` for replacing the stored device ID at `start()` with the given or a newly generated one. The events the stored user left are first sent under the stored device ID. It is meant for testing and forced user resets: left on, every start creates a new user.
 - Added the app version (`av`) and the number of waiting requests (`rr`) to every request.
 - Added the operating system name and version (`_os`, `_os_version`) to the device metrics the SDK sends; values given with `setMetrics` still win.
 - Added event IDs: every event carries its own ID and the current view (`cvid`), custom events the previous custom event (`peid`), view events the previous view (`pvid`) and the platform (`segment`), and crash reports the current view (`_cvid`).
 - Added `setFailedRequestCooldown`: a failed request is tried again after this cool-down, 60 seconds by default, instead of on every update.
 - Added `setRequestDropAgeHours` and the server's "dort" setting for dropping stored requests older than the given age.
-- Added the backoff mechanism: when the server answers slowly, sending pauses for a while. `disableBackoffMechanism()` turns it off, and the SDK Behavior Settings can configure it.
+- Added the backoff mechanism: when the server answers slowly, sending pauses for a while. `disableBackoffMechanism()` turns it off, and the SDK Behavior Settings can configure it. `stop()` doesn't wait for a pause: it sends the waiting requests before the SDK stops.
 - Added gradual trimming of a request queue far over its size limit, at most 100 requests at a time; `disableGradualRequestCleaner()` drops the whole overflow at once.
 - Added `setCrashFilterCallback` for changing or dropping crash reports before they are queued, and `setCustomCrashSegmentation` for segmentation added to every crash report.
 - Added `crash().clearBreadcrumbs()`, `views().stopAllViews()`, and segmentation for `closeViewWithName` and `closeViewWithID`.
@@ -65,6 +66,8 @@
 - Mitigated an issue where values cut to the internal limits were not logged.
 - Mitigated an issue where `setUserDetails` and `setCustomUserDetails` called before `start()` logged that the call was ignored, although the details are sent with the first session.
 - Mitigated an issue where a C++ standard set with `CMAKE_CXX_STANDARD`, for example by a Conan profile, was ignored and the SDK was always compiled as C++14. C++14 stays the default and the minimum.
+- Mitigated an issue where a server URL with a port or a path, such as `https://example.com:8443` or `https://example.com/countly`, was accepted but no request reached the server. A port in the URL now wins over the port argument, with a warning when they differ.
+- Mitigated an issue where user details given before `start()` replaced each other, so that for example custom properties set with `setCustomUserDetails` were lost after a `setUserDetails` call.
 
 ## 26.8.1
 - ! Minor breaking change ! Builds now stop with a clear error when the SDK headers find a nlohmann/json older than 3.11 first on the include path, instead of mixing two versions of the library.

include/countly.hpp

@@ -76,8 +76,9 @@ class Countly : public cly::CountlyDelegates {
   static bool hasInstance(const std::string &name);
 
   /**
-   * Destroys the instance registered under 'name'. Joins its threads, ends its
-   * session and frees its database path claim. A no-op if the name is not
+   * Destroys the instance registered under 'name'. Joins its threads and frees
+   * its database path claim. It neither ends the session nor moves the waiting
+   * events into a request: call stop() first for that. A no-op if the name is not
    * registered. Any reference or pointer to that instance dangles afterwards.
    */
   static void destroyInstance(const std::string &name);
@@ -152,6 +153,11 @@ class Countly : public cly::CountlyDelegates {
 
   void setSha256(cly::SHA256Function fun);
 
+  /**
+   * Makes the app begin, update and end sessions with beginSession(), updateSession() and endSession() instead of the
+   * SDK. While the server's SDK behavior settings contain "Automatic session tracking", that setting decides instead,
+   * also while the SDK runs. Call it before start().
+   */
   void enableManualSessionControl();
 
   void disableAutoEventsOnUserProperties();
@@ -212,11 +218,18 @@ class Countly : public cly::CountlyDelegates {
    * Before start(): the device ID to use when no device ID is stored from an earlier start (SQLite builds keep the
    * device ID in use between starts). A stored device ID wins, and a different ID given here is ignored with a
    * warning, unless enableClearStoredDeviceID() is called too. Without a device ID, start() generates one.
-   * After start(): changes the device ID. With same_user the server merges the old user's data into the new device
-   * ID; without it the new device ID is a new user: the waiting events, and with automatic sessions the session, end
-   * under the old device ID first.
+   * After start(): changes the device ID and chooses by where the device ID in use came from: a device ID the SDK
+   * generated is merged into the new one on the server, as for a user who logs in; a device ID the app gave is changed
+   * without merge, see setDeviceID(value, same_user).
    */
-  void setDeviceID(const std::string &value, bool same_user = false);
+  void setDeviceID(const std::string &value);
+
+  /**
+   * Like setDeviceID(value), but after start() 'same_user' decides: true merges the old user's data into the new device
+   * ID on the server; false makes the new device ID a new user: the waiting events, and with automatic sessions the
+   * session, end under the old device ID first.
+   */
+  void setDeviceID(const std::string &value, bool same_user);
 
   /**
    * Makes start() forget the device ID stored by an earlier start. The events that user left in the database are
@@ -264,8 +277,9 @@ class Countly : public cly::CountlyDelegates {
    * at stop().
    *
    * @param app_key: the app key of your Countly app
-   * @param host: the server URL, an absolute http or https URL such as "https://your.server.ly"
-   * @param port: the server port, 0 for the scheme's default
+   * @param host: the server URL, an absolute http or https URL such as "https://your.server.ly". It may contain a port
+   *              and a path, such as "https://your.server.ly:8443/countly"
+   * @param port: the server port, 0 for the port in the URL or the scheme's default. A port in the URL wins
    * @param start_thread: has no effect, the SDK always starts its update thread
    */
   void start(const std::string &app_key, const std::string &host, int port = 0, bool start_thread = false);
@@ -323,6 +337,12 @@ class Countly : public cly::CountlyDelegates {
    */
   void LocationSettingChanged() override;
 
+  /**
+   * Applies a change of the server's "Automatic session tracking" setting: turning it off ends the session the SDK
+   * runs, and the app's session calls work from then on; turning it on begins a session.
+   */
+  void AutomaticSessionTrackingChanged() override;
+
   /*
    * Checks and returns the size of the event queue in memory or persistent storage.
    */
@@ -914,6 +934,18 @@ class Countly : public cly::CountlyDelegates {
    */
   bool isLocationOn();
 
+  /**
+   * Whether the SDK tracks sessions by itself: the server's "Automatic session tracking" setting while the settings
+   * contain it, otherwise the app's choice with enableManualSessionControl().
+   */
+  bool isAutomaticSessionTracking();
+
+  /**
+   * The port requests go to: the port written in the server URL when it has one, otherwise the port argument. A port
+   * argument that differs from the port in the URL is ignored with a warning, counted as log code w121.
+   */
+  unsigned int serverPort(unsigned int url_port, int port, const char *caller);
+
   /**
    * The shared part of the public calls that give or remove consent: checks the call, expands groups, records w402
    * for unknown names and runs the change for the features whose consent changes.

include/countly/configuration_module.hpp

@@ -50,6 +50,12 @@ class ConfigurationModule : public ConfigurationProvider {
    */
   int getConsentRequired() const;
 
+  /**
+   * The server's "Automatic session tracking" setting ("ast"): -1 while the settings don't contain it, 0 when it is
+   * off, 1 when it is on.
+   */
+  int getAutomaticSessionTracking() const;
+
   unsigned int getRequestQueueSizeLimit() const override;
   unsigned int getRequestDropAgeHours() const override;
   bool isBackoffEnabled() const override;

include/countly/constants.hpp

@@ -238,6 +238,12 @@ class CountlyDelegates {
    * Called by the SDK behavior settings when the server's "cr" setting changes whether consent is required.
    */
   virtual void ConsentRequirementChanged() {}
+
+  /**
+   * Called by the SDK behavior settings when the server's "ast" setting changes whether the SDK tracks sessions by
+   * itself.
+   */
+  virtual void AutomaticSessionTrackingChanged() {}
 };
 
 } // namespace cly

include/countly/countly_configuration.hpp

@@ -42,6 +42,12 @@ struct CountlyConfiguration {
    */
   std::string serverUrl;
 
+  /**
+   * The path of the server URL, such as "/countly" for "https://example.com/countly", which the built-in HTTP client
+   * puts in front of every endpoint. Empty when the URL has none.
+   */
+  std::string serverPath;
+
   /**
    * App key for the application being tracked.
    * Mandatory field.

include/countly/request_module.hpp

@@ -34,8 +34,11 @@ class RequestModule {
   /**
    * SDK central execution call for processing requests in the request queue.
    * Only one sender is active at a time. Requests are processed in order.
+   *
+   * @param closing: true for the last send of stop(), which closes by sending: it doesn't wait for a backoff pause and
+   *                 starts none, so that the waiting requests go out before the SDK stops
    */
-  void processQueue(std::shared_ptr<std::mutex> mutex);
+  void processQueue(std::shared_ptr<std::mutex> mutex, bool closing = false);
 
   void addRequestToQueue(const std::map<std::string, std::string> &data);
 
@@ -46,10 +49,11 @@ class RequestModule {
   void clearRequestQueue();
 
   /**
-   * Replaces the server URL and the port that the following requests go to, the way start() sets them: a URL without
-   * a scheme gets "http://" in front, trailing slashes are removed, and a port of 0 becomes the scheme's default port.
+   * Replaces the server URL, the port and the URL path that the following requests go to, the way start() sets them:
+   * a URL without a scheme gets "http://" in front, trailing slashes are removed, and a port of 0 becomes the scheme's
+   * default port. The built-in HTTP client puts the path, such as "/countly", in front of every endpoint.
    */
-  void setServerUrl(const std::string &url, unsigned int port);
+  void setServerUrl(const std::string &url, unsigned int port, const std::string &path = std::string());
 
   /**
    * Gives every queued request the app key. The caller holds the mutex it passes to processQueue().

src/configuration_module.cpp

@@ -23,6 +23,7 @@ static constexpr const char *KEY_LOCATION_TRACKING = "lt";
 static constexpr const char *KEY_CUSTOM_EVENT_TRACKING = "cet";
 static constexpr const char *KEY_CRASH_REPORTING = "crt";
 static constexpr const char *KEY_CONSENT_REQUIRED = "cr";
+static constexpr const char *KEY_AUTOMATIC_SESSION_TRACKING = "ast";
 static constexpr const char *KEY_SERVER_CONFIG_UPDATE_INTERVAL = "scui";
 static constexpr const char *KEY_LOGGING = "log"; // not used and implemented yet
 
@@ -84,6 +85,8 @@ class ConfigurationModule::ConfigurationModuleImpl {
   std::atomic<bool> crashReportingEnabled{true};
   // -1 while the settings don't contain "cr", otherwise 0 or 1
   std::atomic<int> consentRequired{-1};
+  // -1 while the settings don't contain "ast", otherwise 0 or 1
+  std::atomic<int> automaticSessionTracking{-1};
   std::atomic<unsigned int> eventQueueThreshold{0};
   std::atomic<unsigned int> requestQueueSizeLimit{0};
   std::atomic<unsigned int> sessionUpdateInterval{0};
@@ -274,6 +277,10 @@ class ConfigurationModule::ConfigurationModuleImpl {
       // safe because _onSBSChanged is called outside of _mutex
       _cly->LocationSettingChanged();
     }
+
+    if (changedSettings.contains(KEY_AUTOMATIC_SESSION_TRACKING)) {
+      _cly->AutomaticSessionTrackingChanged();
+    }
   }
 
   nlohmann::json _populateConfigValues() {
@@ -299,6 +306,15 @@ class ConfigurationModule::ConfigurationModuleImpl {
       }
     }
     consentRequired.store(consentRequiredCurrent, std::memory_order_release);
+    const int automaticSessionTrackingVal = automaticSessionTracking.load(std::memory_order_acquire);
+    int automaticSessionTrackingCurrent = -1;
+    if (sdk_behavior_settings.is_object()) {
+      nlohmann::json::const_iterator automaticSessionSetting = sdk_behavior_settings.find(KEY_AUTOMATIC_SESSION_TRACKING);
+      if (automaticSessionSetting != sdk_behavior_settings.end() && automaticSessionSetting->is_boolean()) {
+        automaticSessionTrackingCurrent = automaticSessionSetting->get<bool>() ? 1 : 0;
+      }
+    }
+    automaticSessionTracking.store(automaticSessionTrackingCurrent, std::memory_order_release);
 
     trackingEnabled.store(getBool(KEY_TRACKING, true), std::memory_order_release);
     networkingEnabled.store(getBool(KEY_NETWORKING, true), std::memory_order_release);
@@ -373,6 +389,9 @@ class ConfigurationModule::ConfigurationModuleImpl {
     if (consentRequiredCurrent != consentRequiredVal) {
       changedSettings[KEY_CONSENT_REQUIRED] = consentRequiredCurrent;
     }
+    if (automaticSessionTrackingCurrent != automaticSessionTrackingVal) {
+      changedSettings[KEY_AUTOMATIC_SESSION_TRACKING] = automaticSessionTrackingCurrent;
+    }
     return changedSettings;
   }
 
@@ -394,7 +413,7 @@ class ConfigurationModule::ConfigurationModuleImpl {
         valid = value.is_number_unsigned() && value.get<unsigned long long>() <= UINT_MAX;
       } else if (key == KEY_BOM_RQ_PERCENTAGE) {
         valid = value.is_number() && value.get<double>() > 0 && value.get<double>() <= 1;
-      } else if (key == KEY_TRACKING || key == KEY_NETWORKING || key == KEY_LOGGING || key == KEY_SESSION_TRACKING || key == KEY_VIEW_TRACKING || key == KEY_LOCATION_TRACKING || key == KEY_CUSTOM_EVENT_TRACKING || key == KEY_CONSENT_REQUIRED || key == KEY_CRASH_REPORTING || key == KEY_BACKOFF_MECHANISM) {
+      } else if (key == KEY_TRACKING || key == KEY_NETWORKING || key == KEY_LOGGING || key == KEY_SESSION_TRACKING || key == KEY_VIEW_TRACKING || key == KEY_LOCATION_TRACKING || key == KEY_CUSTOM_EVENT_TRACKING || key == KEY_CONSENT_REQUIRED || key == KEY_CRASH_REPORTING || key == KEY_BACKOFF_MECHANISM || key == KEY_AUTOMATIC_SESSION_TRACKING) {
         valid = value.is_boolean();
       } else if (key == KEY_EVENT_BLACKLIST || key == KEY_USER_PROPERTY_BLACKLIST || key == KEY_SEGMENTATION_BLACKLIST || key == KEY_EVENT_WHITELIST || key == KEY_USER_PROPERTY_WHITELIST || key == KEY_SEGMENTATION_WHITELIST) {
         valid = value.is_array();
@@ -587,6 +606,8 @@ bool ConfigurationModule::isCrashReportingEnabled() const { return impl->crashRe
 
 int ConfigurationModule::getConsentRequired() const { return impl->consentRequired.load(std::memory_order_acquire); }
 
+int ConfigurationModule::getAutomaticSessionTracking() const { return impl->automaticSessionTracking.load(std::memory_order_acquire); }
+
 unsigned int ConfigurationModule::getRequestQueueSizeLimit() const { return impl->requestQueueSizeLimit.load(std::memory_order_acquire); }
 
 unsigned int ConfigurationModule::getRequestDropAgeHours() const { return impl->requestDropAgeHours.load(std::memory_order_acquire); }