1

Changelog

Kshitij

Last Update vor 9 Tagen

v 0.13.41

Added:

1. Preview of the revamped dashboard. New UI is still in development, but core features such as starting a test, test results, regression and performance report and detailed API response diff page are accessible.

2. Support for integrating Gitlab and Bitbucket CI check runs with Hypertest. Users would have to set up the following configuration in the service config:

I. Gitlab: Gitlab base URL(required if Gitlab server is custom hosted), the id of the project which contains the repo, and GitLab personal access token(with API level scope).

II. Bitbucket: workspace name, repo name, username, and app password(with pull request and pipelines scopes).

3. Support for viewing the coverage status of a flow on the Flows page. A new column called "Is Covered" has been added to the flows table, which indicates if the given flow is being covered in any of the test cases captured by Hypertest, to refresh this coverage status on demand a new refresh button has also been added in the actions column of the table.

Changed:

1. Updated documentation for self-managed k8s cluster.

2. Middleware helper functions for encryption/decryption that were added in the last release namely: cryptr, base64Encode, and base64Decode are being replaced by new helper functions with similar functionality. 

Now we have two collections that are provided inside middlewares:

I. npmPackages(collection of selected npm modules, currently contains CryptoJs(JavaScript library of crypto standards) which will be helpful for encryption/decryption of data)

II. htHelperFunctions(collection of helper functions provided by Hypertest, contains: base64Encode, base64Decode)

Fixed:

1. Bug fix for duplicate errors in regression report.

2. Bug fix for downloading Excel reports.

3. Bug fix for the update of flow coverage status in the background job.

4. Bug fix for a max limit of 25 on the test sessions page.

5. Bug fix for empty difference path for VALUE_MODIFIED when the data type of body changes.

Deprecated:

Removed:

Security:

v 0.13.40

Added:

1. Support for checking the existence of the flow while creating it. A button has been added on the create flow dialog, by clicking on it users can get to know if the current flow is being covered in any of the test cases.

2. Support for encryption/decryption and encoding/decoding(base64) of data inside middleware.
Following helper functions have been added to all the middleware:
I. cryptr(collection of encryption, decryption helper functions for popular algorithms e.g., aes-256-cbc),

II. base64Encode(creates a Base64 encoded string),

III. base64Decode(decodes a string of data that has been encoded using Base64 encoding).

3. Added list of top 10 slowest APIs in TEST_COMPLETED notification.

Changed:

1. For STATUS_CODE_CHANGED error, now only candidate status code will be included in the difference path, this is to reduce the number of difference aggregations generated for reporting. Users can opt to include the primary status code in difference path by setting INCLUDE_PRIMARY_STATUS_CODE_IN_DIFF_PATH to "yes" in the service config.

Fixed:

1. Bug fix for HT_EXCEPTION while running a test, fixed logic for checking the content type of a response that was causing this exception.

2. Bug fix for sorting in the Performance report, rows with NA value were shown on top when sorted in descending order.

Deprecated:

Removed:

Security:

v 0.13.39

Added:

Changed:

1. Updated GitHub checks report. Made test info collapsible.

Fixed:

1. Bug fix for ignoring errors from regression report, users were getting an unhandled error while ignoring an error.

Deprecated:

Removed:

Security:

v 0.13.38

Added:

1. Support for integrating Github check runs with Hypertest. To set up check runs users would have to install Hypertest's Github App on their repo and provide the repo URL and installation id in the Service config. Whenever a test is started via CLI the commit id is picked from the environment variables and a check run is created. The check run is completed when the test gets signed off as REJECTED or ACCEPTED.

2. Support for downloading Swagger doc 3.0 from the dashboard. A button has been added to the APIs page.

3. Support for updating the response of a request during a test. Now users can pass updated response object in the output of POST RESPONSE middleware.

Changed:

1. By default, HTML response errors won't be reported by Hypertest, users can change this by setting SHOULD_REPORT_ERRORS_FOR_HTML_RESPONSES to "yes" in the service config.

2. Plain text response errors will be reported.

3. Old sign-offs for tests will be automatically rejected if a new test with the same candidate branch id has been created.

Fixed:

Deprecated:

Removed:

Security:

v 0.13.37

Added:

1. Support for customizing error types that are reported by Hypertest. A new section under settings has been added, from which users can choose which errors to report and define their severities.

2. Support for providing test case ids in the start test CLI command.

3. Support for downloading Test's HTML report from Dashboard.

Changed:

1. Pushed deadline for suspending docker-based Hypertest installations from 1 Sep 2022 to 31 Mar 2023.

2. Updated documentation for setting up self-managed K8s cluster for Hypertest with Microk8s, Longhorn, and Prometheus.

3. Enhanced temporary rule matching algorithm for better performance while running tests.

Fixed:

Deprecated:

Removed:

Security:

v 0.13.36

Added:

1. Added support(Backend) for revisiting changes made to Hypertest's configurations/settings(e.g., Service Config, Middlewares, Manual Assertions, Permanent & Temporary Rules, Deduplication Rules).

UI support for this feature is in development.

2. Added support for defining Flows(test scenarios e.g., login, payment) for application under test. Flows are an ordered set of APIs that represents a user scenario.

A new section for flows has been on the side navigation bar. Users can check for coverage of flows before starting a test and after the test has been completed just like API coverage.

3. Added support for adding user-defined annotations to Hypertest's Ingress deployments with the help of config-map.

Changed:

1. Enhanced reporting for VALUE_MODIFIED errors by omitting common known false positives e.g., changed access-tokens/URLs in the body. Users can turn off this enhancement by setting AUTO_IGNORE_COMMON_VALUE_MODIFICATIONS to "no" in the service config.

2. Improved error reporting when the content type of responses changes. CONTENT_TYPE_CHANGED difference will only be reported if the subtype of content-type has changed e.g., JSON to XML, in such scenarios differences for the body will not be reported.

3. Updated Last Mirrored Request dialog on the Home page, now using Monaco editor to display last mirrored request payload.

4. Changed the icon for adding a new request in a test case.

5. Changed action for updating/restarting Hypertest's Central Service(K8s), now users can do cmmd+click on the dashboard version to open up the dialog.

6. Now service description is a required field while adding new service from Hypertest's Central Service(K8s).

Fixed:

1. Bug fix for secrets when a user tries to save a middleware.

2. Bug fix for performance report, write APIs(e.g., PUT, POST, PATCH) were not included in the performance report if config option SHOULD_TEST_WRITE_REQUESTS is set to "no".

Now write APIs performance stats will be reported for primary application if SHOULD_TEST_WRITE_REQUESTS is set to "no".

3. Bug Fix for service config editor height and scroll.

Deprecated:

Removed:

Security:

v 0.13.35

Added:

1. Support for scheduling a test while HyperTest is busy running another test. Users will be prompted to choose if they want to schedule a test with the selected test configuration when there's already a test in the running state.

2. Support for storing credentials in a secure way for using inside middlewares. A new section called Secrets under settings has been added where users can provide credentials which will be encrypted and stored in the database, these secrets will be accessible inside middlewares. While testing a middleware with secrets users will be asked to provide the secrets which will be stored locally on their browser for easy testing in the future.

3. Support for updating and restarting HyperTest's central dashboard service. Users can do ctrl+click on the central dashboard's version and can choose from a dialog whether they want to update or restart the central service.

Changed:

1. New editor for service config.

Fixed:

Deprecated:

Removed:

1. Removed option for creating a permanent rule with all fields selected as ANY, this would have resulted in ignoring all the differences detected in a test.

2. Option to provide exact match value while creating a permanent rule. Now users can provide a regular expression for matching a value or select ANY while creating a rule.

Security:

v 0.13.34

Added:

1. Support for giving access to your HyperTest instance to external users by creating a short-lived shareable link. A new section for link sharing has been added under settings where users can create or find an active link.

2. Support for downloading database backups and restoring a backup. A new section for back and restore has been added under settings where users can download a backup file and restore a backup file that was previously downloaded.

3. New pop-up notification calling for version upgrade. Users will be notified on the UI whenever a new stable version of Hypertest is available.

4. Support for making Postgres queries inside middlewares. A new connector for Postgres has been added to all the middleware available.

5. Support for sharing variables between sessions at test run time. A new memory store called testRunVariables has been added to the following middlewares: PRE_SESSION, PRE_REQUEST, POST_RESPONSE, POST_SESSION.

This will help while testing a scenario where a user can log in only once and would have to use the newly generated token in every other session in that test case, now you can store this token inside testRunVariables and it will be available to all other subsequent sessions.

6. New API for generating Swagger(3.0.0) documentation. UI support for this feature is in development.

Changed:

1. Revamped API schema dialog on APIs page. Made the user experience resemble Swagger Doc.

Fixed:

1. Resolved error on saving a test case.

2. Resolved errors on Home Page.

3. Fixed autocomplete/suggestions for connectors inside middleware code editor.

4. Fixed logic for deduplication.

5. Fixed APIs for backing up sessions, config, and restoring.

Deprecated:

Removed:

Security:

v 0.13.33

Added:

1. New section added for analytics/troubleshooting, now being served as home page for Hypertest.

2. New notifications for test start failure and pending sign-offs. 

3. Option to turn off deduplication for an API from the deduplication rules page. Selectively turning deduplication off for login APIs would help running tests with requests deduplication on where login APIs are called multiple times.

4. New API for creating a tunnel connection with Hypertest's tunnel server to expose your Hypertest instance by generating a public URL.

5. Option to delete multiple rules on the temporary rules page.

Changed:

1. Now storing session tags with unique tag names. The latest tag value will be kept for a tag name in a session.

2. Changed name for Kubernetes ingress service name for Hypertest's logger service, for better DNS resolution of logger service.

3. Made all searches on UI case insensitive.

4. Revamped UI for API page.

5. Minor UI changes for chat widget and tab title.

Fixed:

1. Redundant API calls from UI on pagination.

2. Fresh installation errors on Hypertest's backend caused by migrations.

3. Fixed copy as curl functionality for PUT/POST requests.

Deprecated:

Removed:

Security:

v 0.13.32

Added:

1. New MEDIUM severity assertion(ARRAY_ORDER_DATA_TYPE_CHANGED) has been added for detecting data type changes when the order of an array has also changed.

Changed:

1. Revamped middlewares UI/UX. New console for logging messages from middleware code for better debugging.

Fixed:

1. Error at starting Hypertest backend caused by migration script for updating request hashes.  

Deprecated:

Removed:

Security:

v 0.13.31

Added:

1. Option to view only those APIs which have failed in a user journey.

2. Run more than one health check before starting a new test.

Added a service configuration option called ADDITIONAL_HEALTH_CHECKS to provide API path, verb, and acceptable status codes for these health checks.

Changed:

1. Admin privilege not required for deploying Hypertest on Kubernetes.

Fixed:

1. Bugfix for saving a test case, the bug was introduced in 0.13.29.

2. Removing a middleware did not take immediate effect.

3. In middlewares fixed handling of requests with content-type 'application/x-www-form-urlencoded'.

Deprecated:

Removed:

Security:

v 0.13.30

Added:

1. New APIs page, lists all the APIs and their unique input schemas captured by Hypertest.

2. Rerun test button, which loads the configuration of a previous test for running that same test again.

3. Response page for the API performance report. Users can view all the responses for a particular API and filter based on the range of status code, response time, and response size and can also do regular expression searches on headers and body.

4. Now storing time taken by Hypertest to calculate differences in API responses and generate reports, currently not visible on UI.

5. New output parameter for middlewares: oneTimeSessionVariables. Keys and values passed in the oneTimeSessionVariables object will be copied to session variables of all the instances, which helps in situations where an auth token or OTP can/should be generated only once.

6. Service configuration option for setting a threshold in minutes(calculated from the completion of a test) after which sing-offs for tests will be automatically rejected.

7. Option for selecting 'ANY' API path or by matching against a regular expression while adding permanent rules for ignoring errors.

8. Service configuration option for reporting noise and unexplained errors. Users can choose to report these errors, by default they will not be reported.

9. Option to select minimum error severity threshold while starting a new test. Only errors that have greater or equal severity to the mentioned threshold will be shown in the regression report.

10. On searching by an API in all test cases page, the searched API path will be highlighted in the Interactions page of the respective test.

Changed:

1. Upgraded version of HTTP client Axios.

2. In the regression report now showing the count of regressions reported based on best-explained errors,

earlier total differences found were shown.

3. Now in case the content type of an API response is changed then instead of reporting both HEADER_MODIFIED and CONTENT_TYPE_CHANGED

only the CONTENT_TYPE_CHANGED error will be reported.

Fixed:

1. Handling of requests with content-type 'application/x-www-form-urlencoded' while running test.

Deprecated:

Removed:

1. From the start new test dialog removed the option for stopping the test on the first failure and running the smoke test.

Security:

v 0.13.28

Added:

1. Option to provide SMTP configuration for sending out Hypertest notifications.

2. A new mind map section, which is a visual representation of APIs relationships with one another. Users can search for one or more APIs and can gain insights into their relations.

3. Option to report an issue on JIRA. Users will be redirected to their JIRA project dashboard with issue details auto-filled.

4. Option to scale logger consumer service of Hypertest from service configuration. Scaling of logger consumers would improve performance while logging requests.

5. POST SESSION middleware, which will be executed after completing the test for a session. Users will be provided with the session's metavariables, axios(HTTP client), and the environment variables and would be expected to return post-session metavariables which will be stored in the session test analysis. This is helpful on [erforming cleanup steps after a session has been run completely

6. Added search filters in the bug tracker. Users will be able to search for issues based on: id, name, comments, status, severity, API path, method, difference type, and difference path.

7. Added webhook payload description in the notification section. 

Changed:

1. Improved Hypertest's performance while logging a request by updating logic for handling request tags.

Fixed:

1. Test progress refresh bug.

2. Now merging duplicate sessions based on a common session differentiator by executing a background job in HyperTest's backend.

This was an issue for high traffic environments 

Deprecated:

Removed:

Security:

v 0.13.27

Added:
Changed:

1. Disabled encoding of cookies in headers of requests while running a test.

Fixed:

Deprecated:

Removed:

Security:

v 0.13.26

Added:

Changed:

1. Improved Hypertest's performance by updating logic for clustering of APIs.

Fixed:

Deprecated:

Removed:

Security:

v 0.13.25

Added:

1. Introduced a new difference type DATA_TYPE_CHANGED(change in the data type of a property in the API response).

2. Introduced an option to sign off/approve all the reported differences in a test. Now teams can easily sign off CI pipeline builds with Hypertest.

3. Option to skip a request at runtime with the help of PRE_REQUEST middleware. Added an optional output parameter shouldSkipRequest for PRE_REQUEST middleware.

4. Option to add permanent rules from regression report with Ctrl + Left Click on any reported changes.

5. Now showing the severity of reported differences in the regression report.

Changed:

1. Updated default function bodies for respective middlewares.

2. Updated database migration logic for faster bootup.

3. Updated documentation for Hypertest cli.

Fixed:

1. Fixed logic for handling cookies.

Deprecated:

Removed:

Security:

v 0.13.24

Added:

Changed:

Fixed:

1. Logic for calculating difference paths for API responses containing arrays of arrays.

Deprecated:

Removed:

Security:

v 0.13.23

Added:

Changed:

1. Improved Hypertests capability of handling large amounts of incoming requests.

Introduced an option to handle scaling and load balancing of Hypertest's logger service with Nginx.

Fixed:

Deprecated:

Removed:

Security:

v 0.13.22

Added:

1. Support for Kubernetes.

2. Option to sort performance report by average latency change(%).

3. Option to expose your service to third-party services with a static URL.

Added a Hypertest CLI command to create a tunnel connection with Hypertest's tunnel server to expose your service by generating a public URL.

Changed:

1. Now by default performance report will be sorted by average latency change(absolute value).

2. Duplicate manual assertion failures across various instances of a candidate's response won't be reported.

Helps in reducing error count in the regression report.

3. Now Request count for APIs in the API coverage dialog can be enabled/disabled on the toggle of a switch.

Fixed:

Deprecated:

Removed:

Security:

v 0.13.21

Added:

1. Now you can inspect all the APIs which would be played in a test, from the API coverage section in the start new test dialog.
API Coverage report will include all the documented and undocumented APIs, only undocumented APIs will be shown in case swagger is not available.

Changed:

Fixed:

Deprecated:

Removed:

Security:

v 0.13.20

Added:

Changed:

1. Now default Primary and Candidate URLs are available in the restest option on the test case interactions page.

Fixed:

Deprecated:

Removed:

Security:

v 0.13.19

Added:

1. Option to run tests on multiple environments.

Environments will be identified by x-ht-env or x-real-host headers.

Changed:

1. Simplified whitelisting of changes. Now you can ignore desired changes with one click from the regression report.

Fixed:

Deprecated:

Removed:

Security:

v 0.13.18

Added:

Changed:

Fixed:

1. VALUE_MODIFIED_EXCEPTIONS to work only on strings.

Deprecated:

Removed:

Security:

v 0.13.17

Added:

1. 10 min cron job to terminate stuck test analysis.

2. VALUE_MODIFIED_EXCEPTIONS in config to avoid reporting differences for certain strings.

Changed:

1. API performance report-related UI changes.

Fixed:

Deprecated:

Removed:

Security:

v 0.13.16

Added:

Changed:

1. Now showing primary and candidate URLs in health check errors.

2. Logger queue messages will no longer be persisted to disk for better performance.

Fixed:

Deprecated:

Removed:

Security:

v 0.13.15

Added:

1. API performance report.

2. Slack notification webhook.

Changed:

1. Middlewares UI.

Fixed:

1. CI issue for not being able to start the test.

Deprecated:

Removed:

Security:

v 0.13.14

Added:

1. Maintenance page link in the dashboard.

Changed:

1. Navigation drawer expand collapse button.

Fixed:

1. Frontend state-related issues.

Deprecated:

Removed:

Security:

v 0.13.13

Added:

Changed:

Fixed:

1. URL validation.

2. Middleware bug.

Deprecated:

Removed:

Security:

v 0.13.12

Added:

1. Option to download test report in excel format from the dashboard and build report.

2. Diff view in responses page.

3. Now using Monaco editor for manual assertion.

4. Feedback widget in UI.

5. Middlewares info page.

6. Common and advanced use cases of middlewares in docs.

Changed:

1. Cron job docs.

2. Now not showing NOT_RUN requests in test results.

Fixed:

1. Saved sessions related bug in Test stats at start new test dialog.

Deprecated:

Removed:

1. OVERRIDE_SESSION_REQUESTS_V1 middleware.

Security:

v 0.13.11

Added:

1. Native cli for CI integration.

3. CI docs.

2. MIDDLEWARE_ENV to htConfig, which will be provided inside middlewares.

3. baseURL for REQUEST_MODIFIER_V1, POST_RESPONSE_SESSION_VARIABLES_V1 and PRE_SESSION_V1 Middlewares.

Changed:

1. UI for test stat at start new test.

2. Now Showing Open Issues count for a test on the dashboard, earlier Total Issues count was displayed.

3. Now Using Monaco editor for Middlewares.

Fixed:

1. Pre Migration Script.

Deprecated:

Removed:

Security:

v 0.13.10

Added:

Changed:

Fixed:

1. PRE_SESSION_V1 Middleware.

Deprecated:

Removed:

Security:

v 0.13.9

Added:

1. Option to deduplicate sessions for a test.

2. Displaying total and unique requests for a test and a user session in stats.

3. PRE_SESSION_V1 Middleware.

4. Running mode config (ONE_DB or TWO_DBS).

5. Warning in the weekly report for missing HT_DASHBOARD_URL.

Changed:

Fixed:

Deprecated:

Removed:

Security:

v 0.13.8

Added:

1. API for CI integration (isTestCompleted)

2. Middleware expected output schema in middleware page

3. Interactions page pagination reversed(now starting from the last page)

4. Difference type information and navigation on interactions page

5. Pre-session middleware

Changed:

1. Build report is being generated after terminating the test

Fixed:

1. Empty sessions issue by filtering sessions before starting a new test

2. Pre-migration script updated

Deprecated:

Removed:

Security:

v 0.13.7

Added:

1. Option to deduplicate requests while running a test

Changed:

Fixed:

Deprecated:

Removed:

Security:

v 0.13.6

Added:

1. Option to ignore errors by retesting from interactions page

2. Option to select the fast mode for tests in Hypertest configuration(Hits requests three times instead of six times)

3. Option to add a description to a test while creating it

4. Option to check test stats in start new test dialog(total session and request count for selected time period and additional test options)

5. API for backup and restore("GET {HYPERTEST_URL}/api/backupConfig", "GET {HYPERTEST_URL}/api/backupSessions", "POST {HYPERTEST_URL}/api/restore")

6. Documentation for starting Hypertest via cron.

7. Hypertest dashboard URL in config for attaching build report URL in test notification email

Changed:

Fixed:

Deprecated:

Removed:

Security:

v 0.13.5

Added:

1. Search test cases by API paths (even without swagger/Open API spec)

2. Option to add a delay before executing requests while retesting

Changed:

Fixed:

Deprecated:

Removed:

Security:

v 0.13.4

Added:

1. Manual Assertions

Changed:

Fixed:

Deprecated:

Removed:

Security:

v 0.13.3

Added:

1. Protobuf support added

2. Option to add a delay before executing requests in a test case

Changed:

1. Coverage report includes past request data as well now

Fixed:

Deprecated:

Removed:

Security:

v 0.13.2

Added:

1. File upload (Multipart) support added

Changed:

Fixed:

Deprecated:

Removed:

Security:

v 0.13.1

Added:

Changed:

Fixed:

1. Autodeletion logic fixed to not delete old saved test cases

Deprecated:

Removed:

Security:

v 0.13.0

Added:

Changed:

1. Probabilistic array comparisons to support data coming from 2 DBs

2. Old path, new path merge into a single diff Path (writing rules will become simpler, 50% less time to query results) (Backwards compatible)

Fixed:

Deprecated:

Removed:

Security:

v 0.12.36

Added:

Changed:

Fixed:

1. autodelete old requests used to delete all test cases older than x days. Now it just deletes requests older than x days

Deprecated:

Removed:

Security:

v 0.12.35

Added: 

Maintenance UI: Accessed via /maintenance/

1. Update Hypertest: Downloads a new version if available

2. Restart Hypertest: Restarts all the Hypertest services.

3. Restart Nginx: Restarts the Nginx container.

Changed:

1. issue tracker: status field added. It can be 'Open' or 'Resolved'

2. 15x faster array comparisons

Fixed:

1. array order change false positives handled

Deprecated:

Removed:

Security:







Was this article helpful?

1 out of 1 liked this article

Still need help? Message Us