ID management changes

CHANGELOG.md

@@ -2,13 +2,15 @@
 - ! Minor breaking change ! The "Consent required" option of the SDK Behavior Settings now applies. While it is on, the SDK collects nothing until the app gives consent; while it is off, the SDK works as if every consent were given, also in apps that require consent.
 - ! Minor breaking change ! The "Automatic session tracking" option of the SDK Behavior Settings now applies. While it is off, the SDK tracks no session by itself and the app's session calls work, as with `enableManualSessionControl()`; while it is on, the SDK tracks sessions also in apps that called `enableManualSessionControl()`. A change while the SDK runs ends or begins the session.
 - ! Minor breaking change ! Every request is now sent as POST, with its data in the request body. A client set with `setHTTPClient` gets `usePost` true for every request and must support POST. `alwaysUsePost` is deprecated and has no effect.
+- ! Minor breaking change ! A client set with `setHTTPClient` must return the parsed JSON answer in `data` for every request: a request counts as delivered only when the answer is a JSON object with a `result`.
 - ! Minor breaking change ! `start()` now refuses a server URL without `http://` or `https://`, without a host, with a port that is not a number from 1 to 65535, or with a query or a fragment, an empty app key and the placeholder `YOUR_APP_KEY`: it logs an error and the SDK stays off. Before, a URL without a scheme was used over plain HTTP.
-- ! Minor breaking change ! Recorded events are now sent on the session update interval (60 seconds by default) or when the event queue threshold is reached, instead of every 3 seconds. `flushEvents()` moves them into a request right away. Feedback answers (`[CLY]_nps`, `[CLY]_survey`, `[CLY]_star_rating`) and `[CLY]_push_action` events are moved into a request as soon as they are recorded.
+- ! Minor breaking change ! The last parameter of `start()` has no effect anymore: with automatic sessions, the SDK always updates the session on the session update interval (60 seconds by default). Before, `start()` without it sent no session updates; for none, use `enableManualSessionControl()`.
+- ! Minor breaking change ! Recorded events are moved into a request on every pass of the update loop, every 3 seconds by default (see `setUpdateInterval`). Before, `start()` without its last parameter held them until 100 were waiting. `flushEvents()` moves them at once.
+- ! Minor breaking change ! `disableSDKBehaviorSettingsUpdates()` now stops every SDK Behavior Settings request, also the one at `start()`, so that only the settings given with `setSDKBehaviorSettings` or stored earlier apply. Before, it stopped only the periodic requests.
 - ! Minor breaking change ! `beginSession`, `updateSession` and `endSession` now work only with `enableManualSessionControl()`; with automatic sessions they are ignored with a warning. A manual `updateSession()` now always reports the session duration, also when called sooner than the session update interval.
 - ! Minor breaking change ! Remote config downloads now use the current remote config request and no longer enroll the user in A/B tests; call `enrollABOnRCDownload()` before `start()` to keep enrolling. With `enableRemoteConfig()`, values are downloaded at the end of `start()`, after a device ID change and when the consent is given, instead of at every session start.
 - ! Minor breaking change ! Config values below their minimum are now ignored with a warning instead of being raised: 0 for the internal limits, `setMaxRequestQueueSize` and `setAutomaticSessionUpdateInterval`, and values below 1 for `setEventsToRQThreshold`. `setEventsToRQThreshold` also keeps values above 10000; the event queue still sends at 1000 events.
 - ! Minor breaking change ! `setLocation` no longer uses values that don't fit their form, such as a lower case country code, coordinates that are not decimal degrees or an invalid IP address; it logs a warning for each.
-- ! Minor breaking change ! SQLite builds now keep the device ID in use between starts, and at `start()` a stored device ID wins: a different device ID given with `setDeviceID` before `start()` is ignored with a warning. To use the given device ID, call `enableClearStoredDeviceID()` before `start()`, or change the device ID with `setDeviceID` after `start()`.
 
 - Added user consent. With `setRequiresConsent(true)` a feature works only after its consent is given, at start with `setConsentEnabled` or later with `giveConsent`, `giveConsentAll` or `setConsent`, and stops when the consent is removed with `removeConsent`, `removeConsentAll` or `setConsent`. The SDK sends every change to the server. The feature names are in `cly::consent`, and `cly::consent::allFeatures()` lists them all. `createFeatureGroup` and `setConsentFeatureGroup` handle groups of features; `getConsent` and `checkAllConsent` show the current state.
 - While consent is required, changing the device ID without merge removes all consent, so the app gives it again for the new user.
@@ -24,10 +26,13 @@
 - Added `disableHealthCheck()` for turning off the health check request.
 - Added `status_code` and `body` to `HTTPResponse`. A custom HTTP client can fill them so that the health check reports the last request the server refused; a client that leaves them empty works as before.
 - Calls made before `start()`, a second `start()`, beginning a session that already runs and ending one that does not are now logged at warning level. Failed and non-JSON server answers, and session calls ignored because the SDK Behavior Settings turned session tracking off, are now logged at debug level.
-- Added device ID generation: when no device ID is stored and the app gives none before `start()`, the SDK generates one. `getDeviceID()` and `getDeviceIDType()` return the device ID in use and where it came from. Every request carries the device ID type (`t`). After `start()`, `setDeviceID(id)` merges a generated device ID into the new one, as for a user who logs in, and changes a device ID the app gave without merge; `setDeviceID(id, same_user)` still decides explicitly.
-- Added `enableClearStoredDeviceID()` for replacing the stored device ID at `start()` with the given or a newly generated one. The events the stored user left are first sent under the stored device ID. It is meant for testing and forced user resets: left on, every start creates a new user.
+- Added device ID generation: when the app gives no device ID before `start()`, the SDK generates one, a new one at every start (see `setClearStoredDeviceID`). `getDeviceID()` and `getDeviceIDType()` return the device ID in use and where it came from.
+- Every request carries the device ID type (`t`); the server counts users whose device ID the app gave as known users, so the dashboard's known user count can grow.
+- After `start()`, `setDeviceID(id)` merges a generated device ID into the new one, as for a user who logs in, and changes a device ID the app gave without merge; `setDeviceID(id, same_user)` still decides explicitly.
+- Feedback answers (`[CLY]_nps`, `[CLY]_survey`, `[CLY]_star_rating`) and `[CLY]_push_action` events are now moved into a request as soon as they are recorded.
+- Added `setClearStoredDeviceID()`. By default `start()` uses the device ID given with `setDeviceID`, or a new generated one. With `setClearStoredDeviceID(false)`, SQLite builds keep the device ID between starts, and a stored device ID wins over one given before `start()` (with a warning).
 - Added the app version (`av`) and the number of waiting requests (`rr`) to every request.
-- Added the operating system name and version (`_os`, `_os_version`) to the device metrics the SDK sends; values given with `setMetrics` still win.
+- Added the operating system name and version (`_os`, `_os_version`) to the device metrics the SDK sends; values given with `setMetrics` still win. Apps that never called `setMetrics` now show their platform and operating system version on the dashboard.
 - Added event IDs: every event carries its own ID and the current view (`cvid`), custom events the previous custom event (`peid`), view events the previous view (`pvid`) and the platform (`segment`), and crash reports the current view (`_cvid`).
 - Added `setFailedRequestCooldown`: a failed request is tried again after this cool-down, 60 seconds by default, instead of on every update.
 - Added `setRequestDropAgeHours` and the server's "dort" setting for dropping stored requests older than the given age.
@@ -49,7 +54,6 @@
 - Mitigated an issue where a negative event count was sent as a very large number. Counts below 1 now become 1, and events with an empty key are ignored, with a warning.
 - Mitigated an issue where an empty device ID was accepted and the server rejected every request.
 - Mitigated an issue where `stop()` left the session open; it now sends the waiting events and ends the session before the SDK stops.
-- Mitigated an issue where sessions were not updated and events were not sent until 100 were waiting when `start()` was called without its last parameter.
 - Mitigated an issue where crash reports lacked the operating system and app version, so the server rejected them. A crash without a stack trace is now ignored with a warning.
 - Mitigated an issue where breadcrumbs stayed after a crash and appeared in the next crash report.
 - Mitigated an issue where an event timed with `startTimer` and `stopTimer` was recorded at its start instead of its end.
@@ -59,7 +63,6 @@
 - Mitigated an issue where an empty blacklist in the SDK Behavior Settings switched off the matching whitelist.
 - Mitigated an issue where events with a `[CLY]_` key the SDK does not know skipped the custom event switch, the event filters and the size limits.
 - Mitigated an issue where `setSDKBehaviorSettings` ignored settings given as a copied server answer, and kept the given settings like a server answer for later starts.
-- Mitigated an issue where `disableSDKBehaviorSettingsUpdates()` still let one settings request go out at start.
 - Mitigated an issue where the periodic settings fetch used the device ID from `start()`. The settings are now also fetched after a device ID change.
 - Mitigated an issue where remote config values downloaded for the previous user stayed after a device ID change.
 - Mitigated an issue where a separate location request carried the IP address.
@@ -68,6 +71,7 @@
 - Mitigated an issue where a C++ standard set with `CMAKE_CXX_STANDARD`, for example by a Conan profile, was ignored and the SDK was always compiled as C++14. C++14 stays the default and the minimum.
 - Mitigated an issue where a server URL with a port or a path, such as `https://example.com:8443` or `https://example.com/countly`, was accepted but no request reached the server. A port in the URL now wins over the port argument, with a warning when they differ.
 - Mitigated an issue where user details given before `start()` replaced each other, so that for example custom properties set with `setCustomUserDetails` were lost after a `setUserDetails` call.
+- Mitigated an issue where events that an earlier start left in the SQLite database were sent under the device ID of the next start.
 
 ## 26.8.1
 - ! Minor breaking change ! Builds now stop with a clear error when the SDK headers find a nlohmann/json older than 3.11 first on the include path, instead of mixing two versions of the library.

include/countly.hpp

@@ -215,9 +215,9 @@ class Countly : public cly::CountlyDelegates {
   void setLocation(const std::string &countryCode, const std::string &city, const std::string &gpsCoordinates, const std::string &ipAddress);
 
   /**
-   * Before start(): the device ID to use when no device ID is stored from an earlier start (SQLite builds keep the
-   * device ID in use between starts). A stored device ID wins, and a different ID given here is ignored with a
-   * warning, unless enableClearStoredDeviceID() is called too. Without a device ID, start() generates one.
+   * Before start(): the device ID start() uses. Without a device ID, start() generates one. SQLite builds that call
+   * setClearStoredDeviceID(false) keep the device ID in use between starts, and a stored device ID then wins: a
+   * different ID given here is ignored with a warning.
    * After start(): changes the device ID and chooses by where the device ID in use came from: a device ID the SDK
    * generated is merged into the new one on the server, as for a user who logs in; a device ID the app gave is changed
    * without merge, see setDeviceID(value, same_user).
@@ -232,12 +232,13 @@ class Countly : public cly::CountlyDelegates {
   void setDeviceID(const std::string &value, bool same_user);
 
   /**
-   * Makes start() forget the device ID stored by an earlier start. The events that user left in the database are
-   * first sent under the stored device ID; then start() uses the device ID given with setDeviceID, or generates a new
-   * one. Meant for testing and for forced user resets: left on in a released app, every start creates a new user.
-   * Call it before start(); after start() it is ignored with a warning.
+   * Whether start() replaces the device ID that SQLite builds stored at an earlier start. On by default: start() first
+   * sends the events that the earlier start left in the database under its device ID, then uses the device ID given
+   * with setDeviceID, or generates a new one. With false, the stored device ID is kept between starts and wins over a
+   * device ID given with setDeviceID, which is ignored with a warning. Call it before start(); after start() it is
+   * ignored with a warning.
    */
-  void enableClearStoredDeviceID();
+  void setClearStoredDeviceID(bool clear);
 
   /**
    * Sets segmentation that every crash report gets under "_custom". The segmentation a recordException call gives
@@ -306,6 +307,10 @@ class Countly : public cly::CountlyDelegates {
    */
   void changeAppKey(const std::string &appKey);
 
+  /**
+   * Sets how often the SDK's update loop runs, in milliseconds, 3000 by default. Each pass moves the recorded events
+   * into a request and sends the request queue; session updates follow the session update interval instead.
+   */
   void setUpdateInterval(size_t milliseconds);
 
   void addEvent(const cly::Event &event) override;
@@ -792,10 +797,10 @@ class Countly : public cly::CountlyDelegates {
   void onMergeDelivered();
 
   /**
-   * Decides the device ID at start(): the one stored by an earlier start, otherwise the given one, otherwise a newly
-   * generated UUID. Returns true when enableClearStoredDeviceID asks to replace a stored device ID: the stored one
-   * stays in use until replaceStoredDeviceId() has queued its user's events. The caller must hold the instance mutex
-   * and the storage must be initialized.
+   * Decides the device ID at start(): the one stored by an earlier start when setClearStoredDeviceID(false) keeps it,
+   * otherwise the given one, otherwise a newly generated UUID. Returns true when a stored device ID is replaced: the
+   * stored one stays in use until replaceStoredDeviceId() has queued its user's events. The caller must hold the
+   * instance mutex and the storage must be initialized.
    */
   bool resolveDeviceId(const std::string &given_device_id);
 
@@ -842,8 +847,8 @@ class Countly : public cly::CountlyDelegates {
   void updateLoop();
 
   /**
-   * One pass of the update loop: once an update interval passed, updates the automatic session and moves the waiting
-   * events into a request; then sends the request queue and stores the health check counters when due.
+   * One pass of the update loop: moves the waiting events into a request, updates the automatic session once an update
+   * interval passed, then sends the request queue and stores the health check counters when due.
    *
    * @param last_update: when the session was last updated by the timer, moved forward when it is updated now
    */

include/countly/countly_configuration.hpp

@@ -120,9 +120,10 @@ struct CountlyConfiguration {
   bool enrollABOnRCDownload = false;
 
   /**
-   * Whether start() replaces a device ID stored by an earlier start with the given or a newly generated one.
+   * Whether start() replaces a device ID stored by an earlier start with the given or a newly generated one. On by
+   * default; setClearStoredDeviceID(false) keeps the stored device ID between starts.
    */
-  bool clearStoredDeviceID = false;
+  bool clearStoredDeviceID = true;
 
   /**
    * Segmentation added to every crash report.

src/countly.cpp

@@ -1225,12 +1225,12 @@ bool Countly::resolveDeviceId(const std::string &given_device_id) {
   configuration->deviceIdType = stored["type"].get<int>() == static_cast<int>(DeviceIdType::SDK_GENERATED) ? DeviceIdType::SDK_GENERATED : DeviceIdType::DEVELOPER_SUPPLIED;
   session_params["device_id"] = configuration->deviceId;
   if (configuration->clearStoredDeviceID) {
-    log(LogLevel::INFO, "[Countly] start, The device ID stored by an earlier start is replaced, as asked with enableClearStoredDeviceID: [" + configuration->deviceId + "]");
+    log(LogLevel::DEBUG, "[Countly] start, The device ID stored by an earlier start is replaced: [" + configuration->deviceId + "]");
     return true;
   }
   if (!given_device_id.empty() && given_device_id != configuration->deviceId) {
     log(LogLevel::WARNING, "[Countly] start, The device ID stored by an earlier start is used: [" + configuration->deviceId + "], and the device ID given with setDeviceID is ignored: [" + given_device_id +
-                               "]. To use it, call enableClearStoredDeviceID() before start(), or change the device ID with setDeviceID after start().");
+                               "] because setClearStoredDeviceID(false) keeps the stored one. To use it, leave the stored device ID cleared, or change the device ID with setDeviceID after start().");
   } else {
     log(LogLevel::DEBUG, "[Countly] start, Using the device ID stored by an earlier start: [" + configuration->deviceId + "]");
   }
@@ -1263,13 +1263,13 @@ void Countly::useNewDeviceId(const std::string &given_device_id) {
   storeDeviceId();
 }
 
-void Countly::enableClearStoredDeviceID() {
+void Countly::setClearStoredDeviceID(bool clear) {
   if (is_sdk_initialized) {
-    log(LogLevel::WARNING, "[Countly] enableClearStoredDeviceID, This method can't be called after SDK initialization. Returning.");
+    log(LogLevel::WARNING, "[Countly] setClearStoredDeviceID, This method can't be called after SDK initialization. Returning.");
     return;
   }
   std::lock_guard<std::mutex> lk(*mutex);
-  configuration->clearStoredDeviceID = true;
+  configuration->clearStoredDeviceID = clear;
 }
 
 void Countly::storeDeviceId() {
@@ -3028,13 +3028,12 @@ void Countly::updateLoop() {
 
 void Countly::updateTick(std::chrono::steady_clock::time_point &last_update) {
   if (!stopping) {
+    packEvents();
     const std::chrono::steady_clock::time_point now = std::chrono::steady_clock::now();
     if (now - last_update >= std::chrono::seconds(configurationModule->getSessionUpdateInterval())) {
       last_update = now;
       if (isAutomaticSessionTracking()) {
         updateSessionInternal(true);
-      } else {
-        packEvents();
       }
     }
   }