======================================================= Z-Wave Compliance Test Tool (CTT) - Release Information ======================================================= If you encounter any problems with the CTT, find any bugs or encounter any crash, please contact: certadmin@z-wavealliance.org or support@mk-logic.de CTT 3.7.1 (2024-07-25) ====================== NEW: - CTT Auto-Update has been implemented. - Double-clicking CTT project files will open them within CTT. CHANGED: - Updated CTT components including fixes and new tests in accordance with cert program 2024A to the following versions: - ZWP Resources: v7 (including DUT-based 'Mini-Form' updated for 2024A) - ZATS: v9 - ZATS Cert Tests: v14 - CTT Scripts: v34 (minimum CTT version 3.7 due to breaking changes within Z-Wave XML) - Updated Z-Wave XML to version 2.14.6 (ZWaveDLL, ZATS, Official CTT Scripts Solution) including generated C# classes (skipped the broken ones which are to be fixed in Z-Wave XML and/or its editor / generator). - DUT Interview: Added warnings when advertised security CCs do not match supported security schemes. - CTT user scripts: Reworked script compiling and execution to focus error view in case of any compile errors. - CTT scripts differ: Save and restore position and size of diff window. - View of 'Mini-Form' in DUT-based projects: Moved buttons. - Implemented progress bar when reflecting all ZATS tests (only the case when updating this component while the meta data list is missing). - Re-arranged portal connections. - Reworked documentation regarding required hardware. Improved description for Zniffer PTI (still needed for Long Range in this CTT version). - Updated license information and extended installer to distribute this file. - Package maintenance: Changed suggested export destination for 'Official CTT Solution' packages to a separate folder. Made exported 'Version XML' considering CTT-internal values for portal connections as source of truth rather than those downloaded from a server. FIXED: - Fixed exception in 'Test Case Log History' when opening a foreign project. - Fixed exception when using 'Advanced Inclusion' dialog. - Fixed CTT not being able to access component downloads via SAML. - Fixed typo in CTT resources path. - Fixed using correct 'ZATS Tests' package version during components update. - Script maintenance: Fixed a bug where new original scripts could not be added to the 'Official CTT Script Solution'. - Fixed version string in 'About' dialog considering alpha and beta versions. KNOWN ISSUES: - For CTT hardware, and in particular for the CTT End Device, the CHIP MUST BE ERASED before flashing the Serial API firmware. - Since CTT 3.4 / 800 series End Device firmware 7.18 occasionally the CTT / ZATS End Device does not answer a KEX Get during S2 bootstrapping so it times out. In the majority of cases this is not reproducible and will work when running the test again. Please make sure to have installed the latest security firmware on the 800 series development board: In Simplicity Studio select the dev board > Overview > General Information > Secure FW. - In rare cases the CTT Script engine gets into a state where one received command from the DUT is taken for multiple subsequent EXPECT statements. If this happens, please reset and re-include the DUT in 'Project Properties' > 'Configure Network' and run the script again. - Trying to set the RF region to UZB7 sticks flashed with controller firmware 7.19 or newer may cause a failure message, even if it actually worked. The recommended firmware for CTT devices is provided in '[User]\Documents\ Z-Wave Alliance\Z-Wave CTT 3\CTT Device Firmwares'. - If the DUT interview data is missing or incorrect during project creation or update, scripts and few ZATS tests which only run on a particular Root Device / End Point if a Command Class is supported, those tests cannot be automatically added to the CTT project if the Command Class is not advertised on the respective End Point. It may also happen that scripts for CCs listed in the ZCP Form are added to the root device while not supported there. The option 'Project' > 'Repair Project' can be used to consider the recent DUT data to add or remove those tests. - CTT Controller as a test tool will only advertise support for a very limited minimum of CCs, unless used within a ZATS test run (e.g. no Version CC, no Supervision CC during inclusion and script runs). - Rarely, the DUT interview does not learn all supported / granted security schemes. In this case, start the interview (or inclusion) again. The issue mostly happens if the DUT answers relatively slowly so that Security Learning takes more than a second. - Hardware detection for Zniffer PTI will offer all development boards available via IP. It cannot be determined whether the Zniffer PTI firmware is indeed flashed (before actually using it and seeing detected Z-Wave frames). Also, it cannot be detected if the board is already used by another application. Only one connection is possible. Be careful, the last application which connects will get the traffic. - The selection of scripts and their sequences are synchronized between Classic and Long Range tabs and cannot be saved individually. - If a CTT project is transformed from a non-Long Range project into a Long Range project, results of Classic scripts are also reset to 'pending'. - If a script execution is stopped, the current test sequence gets the 'passed' state instead of 'pending' or 'aborted'. - CTT projects shall not be saved within actively syncing cloud service folders (like Dropbox). Cloud synchronization may block tests from writing log files which causes issues like unexpected test behavior. - Loading long logs into the Test Reporter might take quite some time. - Script editor: Find/Replace function only finds statements below the cursor and only the first occurrence in a line. - Form View: Portal result status 'n/a' is shown differently in CTT. - Test Reporter is closed when uploading results. - Advanced Inclusion: Deselecting 'Is SUC Capable' does not work as expected. - DUT-based projects: S0-only nodes are not considered. S2 tests will still be added. - MKL beta server URL must not end with a slash '/'. POSSIBLE ISSUES SEEN IN PREVIOUS VERSIONS: - 700 series UZB sticks with a 7.13 firmware will not work correctly if power save mode is switched on in Windows device manager settings. They fall to sleep after a few seconds and will not wake up anymore without reconnecting them. As a workaround disable the following option: Device Manager > Ports (COM & LPT) > Silicon Labs CP210x USB to UART Bridge (COMxx) > Properties > Power Management > Allow the computer to turn off [disabled]. However, even then, UZB7 sticks must be reconnected once after starting the computer. In any case it is highly recommended to use the firmware for the CTT hardware provided in '[User]\Documents\Z-Wave Alliance\Z-Wave CTT 3\ CTT Device Firmwares'. - It happened on startup that CTT did not get past the Simplicity Commander Check at startup. Most probably this was an UI issue only. If restarting CTT does not solve it, the user config, possibly inherited from older CTT versions, should be checked in '[User]\AppData\Local\Z-Wave_Alliance\ ZWaveCTT.exe_\[3.x.x.0]\user.config'. CTT ZATS 9 / ZATS Cert Tests 14 [2024A] (2024-07-25) ---------------------------------------------------- NEW: - Implementation of Schedule Entry Lock CC support emulation. - Implementation of User Credential CC support emulation. Includes interactions with Door Lock, Schedule Entry Lock and User Code CC support emulations. - Introduced 'SpecVersion' and 'CertProgramVersion' for ZATS test cases (used in new and adjusted tests). - CCT_CRC16EncapsulationCmdClassV1_Rev02 derived from Rev01. Reworked to automate or skip some sub-tests if tested elsewhere already. - CSR_AssociationsSecurityLearning_Rev01 created for Av4/MCAv5, derived from Av3/MCAv4 test CSR_ControlViaAssociationSecurityLearning_Rev01. - CSR_AssocV4OrMCAV5IsSupported_Rev01 created, derived from CSR_AssocV3OrMCAV4IsSupported_Rev01. CHANGED: - Merged ZATS and Z-Wave DLL from Z-Wave End Device Emulator (EDE) into CTT containing all the adjustments made for the 'Command Class Support Implementations' (emulations) in EDE. This affects all tests where a CTT (end) device makes use of these CC emulations. - Improved log text when data of Static Controller is missing. - Association and Multi Channel Association CC support emulations: Increased maximum selectable CC versions to Av4/MCAv5 (even though automatic Security Learning had not been implemented). - Barrier Operator CC support emulation, CCR_BarrierOperationCC_Rev01/02/03, CSR_LifelineMandatoryReports_Rev01 and 02: Adapted breaking changes of Z-Wave XML 2.14 for Barrier Operator CC command names. - CCR_BarrierOperatorCC_Rev03: Few delays added. - CCR_Security2CC_Rev02: Deprecated manual CCR test for Security 2 when already tested elsewhere. - CCR_UserCodeCC_Rev04: Show both User ID and current User Code in log when asking to modify the existing User Code as clarified by AWG. Added back testing a new User Code but on any unused User ID, in contrast to modifying existing ones. - CDR_UseOfMultiCommandEncap_Rev01: Skipped unnecessary sub-tests. - CR_AssignReturnRoutesIfSISIsPresent_Rev01: Skipped when already tested for CSC (Inclusion Controller side). - CR_LifelineDestinationNodeId_Rev01: Skip test for CSC and RPC since already tested in other tests. - H_RangeTestLongRange_Rev01: Clarified that the LR range test is meant for all LR channels. - RT_CSCLifelineConfigIfNotSIS_Rev01/02, RT_CSCLifelineConfigIfSIS_Rev01/02, RT_CSCMustAcceptSISRole_Rev02/03: Adapted breaking changes of Z-Wave XML 2.14 for Assign SUC Return Route command field name 'NodeID'. - S2_PrimaryMayPerformS2Bootstrap_Rev01: Consider form selection 'Yes' or 'No' if available. - SSR_CanBeIncludedUsingSmartStart_Rev02 and SSR_CanPerformSmartStartInclusion _Rev01: Write key results from actual SmartStart tests in order to replace the Form integrity test (RT_SupportedSmartStartFunctionality_Rev01). - SSR_ResetAfterS2BootstrapError_Rev03: Increased expect timeouts by 20 seconds. - Manual tests referring to Wiki pages are using new Wiki URL structure. FIXED: - User Code support emulation: Fixed duplicated User Code handling. - CCR_MeterCC_Rev01/02: Fixed scales within unsolicited Meter Reports. - CCR_MeterCC_Rev02: Fixed unsolicited report after Meter Reset not sending to trigger node. - CCR_UserCodeCC_Rev03/04: Fixed supporting node to not have duplicated user codes. - CCR_WakeUpCC_Rev01/02: Made sure the CTT End Dev's Wake Up destination is not set before including it. - CDR_UseOfMultiCommandEncap_Rev01 result handling fixed. - CSR_AssocV3OrMCAV4IsSupported_Rev01: Fixed requirement text. Consider changed CCs as input key. - CSR_LifelineMandatoryReports_Rev02, CSR_LifelineReportToMultipleNodes_Rev01 and CSR_LifelineReportTriggeredByZWave_Rev01: Door Lock CC: Fixed configuring door handles that had no impact. Use Operation Type = Constant for both preparation and trigger commands. - S2C_02563_WrongKEXReport_Rev01/02/V2_Rev01: Fixed verification of #2 tests (DUT includes CTT) which had always been passing. - S2C_02572_MsgEncapMPANExtension_Rev01, S2C_02573_MsgEncapMPANExtMoreToFollow _Rev01, S2C_02576_MsgEncapMOSExtension_Rev01: Prevented possible exceptions. - S2C_02574_MsgEncapMPANExtGroupID_Rev01: Fixed test setup so that both CTT controllers know of and understand each other even with newer serial API versions. - S2C_02575_MsgEncapMGRPExtension_Rev01: Fixed steps 8/9/10 which did not verify anything if no re-sync had been requested at all. Fixed order of verification: Prefer S2 Msg over S2 Nonce. Prevented possible exceptions. - S2C_02583_NetworkKeyGet_Rev01: Replaced not working S2 Multicast test with actual non-secure test. Improved workflow. Added possibility to skip steps for testing. - S2C_02600_BootstrappingInterruptPoints_Rev01: Fixed KEX FAIL (Cancel) command in step 1.8 and send it when S2 bootstrapping is already ongoing. Removed non-answering CRC16 CC from CTT Controller. KNOWN ISSUES: - General: CRC16: The ZATS Controller does not answer as asked. - General: If a Controller-DUT does not have the CSC Role Type, some tests requiring the DUT to perform S2 bootstrapping might not work correctly because the DUT must assign the SIS role to the CTT Controller which has CSC RT so that the DUT (i. e. Real Primary) becomes an Inclusion Controller instead of staying Primary. - General: Some tests do not test S0-only devices correctly but expect S2 to be supported. - Long Range: LR ZATS tests require Zniffer PTI. Using the serial LR Zniffer is not yet implemented in this ZATS version. - CPD_ConfigurationParameterData_Rev01: Does not capture reports to follow if sent very fast. POSSIBLE ISSUES SEEN IN PREVIOUS VERSIONS: - Z/IP Gateways with 7.13.1 firmware or older might experience S2 synchronization errors due to decryption errors. It is highly recommended to use a newer Z/IP Gateway version. - When using CTT to test Controller-DUTs based on Z/IP Gateway in some tests (e.g. S0_ControllerCanIncludeS0Devices_Rev01) Nonce Gets from the DUT may not be answered directly after bootstrapping. This happens because Z/IP Gateway starts the device interview too quickly and before the Test Case is ready. In this case try to run the Test Case again or test the requirement manually. CTT Scripts 34 [2024A] (2024-07-25) ----------------------------------- NEW: - AssociationCmdClassV4_Rev01 created, derived from V3 Rev04. - BatteryCmdClassV1_Rev07/V2_Rev06/V3_Rev05: New revisions considering that as of 2024A the battery level 0xFF (low battery) must only be sent unsolicited. Also implemented retries for 'Interactive_WaitForBatteryLowReport'. - IndicatorCmdClassV3_Rev10/V4_Rev06: New revisions. Refactored 'IndividualValuesNodeIdentify' as standalone test considering recommended report values for set value 0x00 in property 5 ('On time within an On/Off period') as clarified in spec version 2024A. - MultiChannelAssociationCmdClassV5_Rev01 created, derived from V4 Rev06. - UserCredentialCmdClassV1_Rev01: Draft of User Credential CC script is work in progress. CHANGED: - BarrierOperatorCmdClassV1_Rev01/02/03: Adapted renamed command names 'EventSignalSet', 'EventSignalingGet' and 'EventSignalingReport' within script revision each due to breaking changes in Z-Wave XML 2.14. KNOWN ISSUES: - BasicMappingForWinCovPosAndEpAwareZWPv2_Rev01 contains not test sequences. Do test manually instead. - Multilevel Switch or Color Switch scripts with 'Duration' test sequences must be calibrated but may still be too strict. If they are testing recommended durations, try to get as close as possible according to the DUT’s hardware capability. Then consider it passed. - NodeNamingandLocationCmdClassV1_Rev01: Should not test ASCII char 0x00 (NUL). - UserCodeCmdClassV2_Rev05: DUT must support 10 digit codes. But if it does not, the error message may be misleading. - WakeUpCmdClassV2_Rev10/11: When DUT Interval Step is 0, Supervision test sequence may choose a 'not allowed' interval value which actually is allowed. In this case, use a 'not allowed' value manually. CTT 3.6.1 (2024-01-04) ====================== NEW: - Added capability to transfer Product Name, Model Number and Cert ID from CTT to ZATS tests. CHANGED: - Updated CTT ZATS Tests to version 12 and CTT Scripts to version 33 including fixes and new tests in accordance with cert program 2023B. - Increased RequestTimeoutMs in Z-Wave DLL (NetworkViewPoint) to more than one second. Affects both CTT and ZATS tests. This should fix the sometimes occurring issue where some steps of Security Learning were aborted. - Improved DUT Interview: Get Z-Wave Software Report (if any) for 'Protocol' and 'Application' sub-sub versions each. Distinguish requested security schemes vs. finally supported security schemes and active security scheme. - Use default exclusion mode within CTT (like in ZATS). - Use Z-Wave Alliance logo instead of Z-Wave logo. - Show up to 24 recent projects. - Use the user’s credentials of the respective cert portal when looking for CTT updates. FIXED: - Fixed updating ZWave_CTT_CommandClasses.cttxml. - Fixed considering that primary vs. alternative update URL are not necessarily based on ZWA ZCP vs. Legacy Portal. Use centrally maintained server URLs. CTT ZATS 7 / ZATS Cert Tests 12 [2023B] (2024-01-04) ---------------------------------------------------- NEW: - CCR_NotificationCC_Rev04: New revision. Added testing new events as of 2023B. Added testing unknown types and events. Added testing recommended ability to configure rules based on notifications. Added testing recommended capability to update list of known notifications. - CCR_UserCodeCC_Rev04: New test in accordance with 2023B interview requirements (CL:0063.01.21.02.3, CL:0063.01.21.03.1, CL:0063.01.22.01.1). - CR_MinimumControlFunctionality_Rev02: New test revision considering the test is now always selected in the ZCP Form for controllers. - DT_IRRepeaterNoAddSuppCCs_Rev01 and DT_RepeaterNoAddSuppApplicationCCs_ Rev01 inherited from DT_MandatoryNotSupportedCCs_Rev01. DT_RepeaterNoAddSuppApplicationCCs_Rev01: Check for Application CCs, allow others. - PI_CertificationData_Rev01 created for Certification Data comparison. - RD_SmartStart_Rev02: New revision for 2023B. - S2_NLSNotDeactivatable_Rev02: New revision in order to replace a duplicated requirement number in accordance with spec version 2023B. CHANGED: - General: Show full DSK of joining node during inclusion. - General: Whitespace: Removed non-breaking space. - CC Support Utils: Reworked and added method for getting human readable names for notification types and events also considering unknown (not yet defined) values. - Notification Support: Added new events. Allow emulating unknown events. FIXED: - Base class: Sort security schemes in human readable string. - CCA_NotificationCmdClassBackwards_Rev01/..V8Backwards_Rev01: Fixed test case group. - CCR_NotificationCC_Rev01, Rev02, Rev03: Fixed typos and adapted changes in CC Support Utils. Rev03: Fixed verification of Alarm Get (Pull nodes). - CCR_UserCodeCC_Rev03: Added missing observation of v1 User Code Set in v2 test sequence. Made sure user code slots are not empty for User IDs which shall be modified by the DUT later on. Fixed possible exceptions. Added missing requirement number. - CCR_SoundSwitchCC_Rev02: Fixed requirement number. - CR_MinimumControlFunctionality_Rev01: Fixed verification in case there are no controlled CCs found in the ZCP Form. - CSR_LifelineMandatoryReports_Rev02: Skip Central Scene Configuration Report when not applicable. - CSR_LifelineReportTriggeredByZWave_Rev01 and CSR_LifelineReportToMultipleNodes_Rev01: Fixed MCA Set by emptying the NodeID field (before the marker) in order to remove the duplicated entry. Reworked verification of inclusions. Removed Association in favor of MCA. Explicitly added return route assignment for ZatsEndDev to DUT. - DT_MandatoryNotSupportedCCs_Rev01 fixed: Test all End Points. Consider matching Device Types only. - RT_CSCMustAcceptSISRole_Rev03 and 02: Fixed workaround for avoiding Supervision-encap for frame watching by Zniffer. Increased Zniffer timeout. Rev03: Wait for DUT to be ready before requesting Add Mode. - S0_DelaySchemeReportAfterSchemeInherit_Rev01: Wait manually before sending S0 Cmds Supp Get. - S0_EndDeviceCanBeS0Included_Rev01: Zniffer timeout increased. - S2_AbortBootstrapWithoutUnauth_Rev01, S2_GrantedS2UnauthClass_Rev01, S2_SISMaySupportCSA_Rev01 fixed which sometimes had no result when DUT data did not match: ZCP-based: Form Fail. / DUT-based: Pass (does not apply). - S2_UnauthLearnModeECDHKeyPair_Rev01: Clarified test description and fail messages. CTT Scripts 33 [2023B] (2024-01-04) ----------------------------------- NEW: - Battery scripts, new revisions: Workaround added in 'Interactive_WaitForBatteryLowReport' in order to not consider Battery Reports before having clicked 'OK'. - CentralSceneCmdClassV3_Rev07: 'Interactive_CheckSupportedScenes' and 'Interactive_CheckSlowRefresh': Workaround added sending / expecting ZWP Info Get / Report in order to not confuse the next expected Central Scene Notification with the previously expected one. - NotificationCmdClassV8_Rev19: New script revision containing new events in accordance with 2023B. FIXED: - Meter v2–v6 scripts: Increased global load delay to a default value of 60 seconds within recent revision each. - UserCodeCmdClassV2_Rev05 + V1_Rev08: Fixed field names for Remaining Lock Time in Door Lock Operation Report in accordance with Z-Wave XML 2.13.0. CTT 3.5.2 (2023-07-25) ====================== +---------------------------------------------------------------------------+ | Important notice when upgrading to version 3.5 | | '''''''''''''''''''''''''''''''''''''''''''''' | | | | Due to a new installer it is recommended to manually uninstall older CTT | | versions before installing version 3.5. | +---------------------------------------------------------------------------+ NEW: - Tests have been added in regards to the 2023A certification program update. - Command Classes have been added to ZATS and Z-Wave DLL in accordance with Z-Wave XML 2.13.0. - An option to manually repair a project considering the recent DUT interview data has been added to Project > Repair Project. If the DUT interview data was missing or incorrect during project creation, this option can be used to automatically add or remove tests which require correct advertisement of CCs in the DUT data for CTT to know on which End Point those tests belong (in particular tests from the 'Scripts' group). - Script import now has an option to offer all (older) revisions. - A Log History feature has been added to the Test Reporter. CHANGED: - Switched to NSIS installer. - Previous DUT data is restored when inclusion / interview is aborted. - Remember up to 18 recently opened projects. - Within CTT the serial Zniffer will only still use the classic counterpart region if the desired LR region is not supported by the particular Zniffer firmware. (Serial LR Zniffer is not implemented for ZATS tests, yet.) - Warn (not error) if script run is started without any DUT. - Update servers adjusted to Z-Wave Alliance Portal server rather than Legacy Portal server. - Provide portal URLs to MKL beta servers in CTT alpha and beta versions. FIXED: - Z-Wave DLL, Network View Point: Request timeout has been decreased in order to prevent Wake Up nodes from already sleeping when continuing communication. - Fixed exception in DUT vs. Form Comparison when the DUT advertises an unknown combination of Device Classes on End Point 1 or higher which does not match any known ZWP or ZWPv2 Device Type. - Fixed exception in DUT vs. Form Comparison when the ZCP form template contains empty values for a specific Command Class. - Fixed exception when replication was done using the advanced inclusion menu. - Fixed possible exception when removing a script while the Test Case Explorer tree was loading. - Documentation: Old double-click functions in Test Case Explorer have been removed. - Replaced all remaining occurences of 'Silicon Labs' with 'Z-Wave Alliance'. - Fixed misleading DUT interview log output for Central Scene CC with identical scenes. - Fixed manually stopping script execution while waiting for Wake Up Notification. - Fixed an issue where sometimes user scripts were not actually saved in an LR project. - Duplicated LR scripts are no longer displayed in the diff menu. - Fixed a possible exception with tool tips. - Numerous fixes and improvements for existing Test Cases (ZATS tests and CTT scripts) have been implemented. CTT 3.4.1 (2023-01-30) ====================== +---------------------------------------------------------------------------+ | Important notice when upgrading to version 3.4 | | '''''''''''''''''''''''''''''''''''''''''''''' | | | | Projects created or saved with CTT v3.4 are not compatible with older CTT | | versions. Whereas, projects created with versions older than 3.4 can be | | loaded in CTT v3.4 and will be automatically transformed into recent | | projects. | | | | Due to substantial changes of the test environment most test results of | | existing projects created with older versions will be reset to 'pending' | | when loaded in CTT v3.4. Thus, it is recommended to not switch to CTT | | v3.4 during an ongoing Self-Certification. | +---------------------------------------------------------------------------+ NEW: - Test Cases (ZATS tests and CTT scripts) have been added in regards to the 2022B AWG and CSWG specifications and certification program update. - Z-Wave Long Range end devices (DUT) can be included by the CTT Controller via SmartStart Long Range inclusion. - Zniffer PTI is added and needed as Zniffer in a Long Range network. - Firmware for Zniffer PTI firmware has been added. - Former manual Long Range tests for end devices have been implemented as 'Inclusion' and 'Automatic' tests. - CTT is able to detect serial APIs even if sleeping (e.g. emulated non-listening end device) by trying to soft reset the dev board. - New revisions are added to DUT-based projects. CHANGED: - CTT's project system has been reworked in regards to the Long Range integration for end devices. If the DUT has been selected (ZCP Form-based or DUT-based in CTT) to be a Long Range end device, the CTT project will become a Long Range project and require certain tests to be run in either classic or Long Range mode, or both. The 'Classic Tests' and 'Long Range Tests' tabs have been added as parent tabs - each contains the known test groups 'Automatic', 'Interactive' etc. as child tabs. Recent inclusion state and respective required hardware are considered. - Exclusionary terms have been removed within CTT, ZATS, Z-Wave DLL, Scripts and recent ZATS Test Cases - also technical names, e.g. end node Role Types. Their previous terms have been kept in old Test Case revisions in order to match the respective old version of the specifications and certification form revisions. - ZATS (Z-Wave Automated Test System) and Z-Wave DLL have been updated. - Provided firmware files (serial API and bootloaders) for CTT devices have been updated for 700 and 800 series (7.18.1). FIXED: - Fixed an issue where for some 'Interactive' tests the security key folder in the log had remained empty. - Fixed adding DUT-based Test Cases with Device Type filter. - The update message box will stay in foreground. - Numerous fixes and improvements for existing Test Cases (ZATS tests and CTT scripts) have been implemented. CTT 3.3.1 (2022-06-30) ====================== NEW: - Test Cases (ZATS tests and CTT scripts) have been added for the 2022A specification and certification program update. - Support for CTT Controller(s) and CTT End Device(s) using serial API on 800 series hardware has been added. - CTT log and button control strings are mirrored to standard output (stdout). - New revisions added to DUT-based projects. CHANGED: - Less CTT hardware is required for 'S2 Certification': only two CTT Controllers (at least one bridge) and two CTT End Devices (before three and three). - Improved and equalized log and buttons. - ZATS (Z-Wave Automated Test System) and Z-Wave Command Class XML have been updated. - Provided firmware files (serial API) for CTT devices have been updated for 500 series (6.84.00 [6.10]) as well as 700 and 800 series (7.17.2). Bootloaders have been added. FIXED: - Previously known issue with controller serial API 7.15+ has been fixed. The Node Base Type is now set at every serial API restart in order to support both 8-bit and 12-bit Node IDs. - Test Reporter: Logs are now opened being scrolled up to the beginning. - Fixed lookup for mandatory Command Classes vs. Device and Role Type matrix. Fixed Sound Switch DTv2 lookup. - Documentation: Fixed to be available offline. Content updated referring to Simplicity Studio v5 as well as for 800 series devices. - Numerous fixes and improvements for existing Test Cases (ZATS tests and CTT scripts) have been implemented. CTT 3.2.4 (2022-02-03) ====================== CHANGED: - QR Code Data is kept in Project Properties when re-including the same device. - Test Reporter: Minimum size for Test Case Input and Results added. - Form vs. DUT Compare is now showing the v1 Command Class name instead of 'Unknown' if the advertised version is not existent but the CC exists. - Using serial APIs on 800 series hardware as CTT Controller or CTT End Device is not implemented, yet. Thus, in this CTT version serial devices with 800 series chipsets have been removed from the list of selectable serial devices. (800 series DUTs can be tested though.) FIXED: - Update process for Script updates fixed. - Fixed CTT device advertisement for Inclusion Controller Command Class within Test Cases where DUT has Learn Mode and is Inclusion Controller. - Potential issue fixed when connecting to the CTT Controller too often. - Fixes and improvements for existing Test Cases and Scripts have been implemented. CTT 3.2.3 (2021-12-21) ====================== +---------------------------------------------------------------------------+ | Important notice when upgrading to version 3.2 | | '''''''''''''''''''''''''''''''''''''''''''''' | | | | Since version 3.2 CTT is a Z-Wave Alliance tool. Installation, settings | | and documents folders have been adjusted from 'Silicon Labs' to 'Z-Wave | | Alliance'. Settings from CTT v3.1 are automatically transferred to the | | CTT v3.2+ version. | +---------------------------------------------------------------------------+ NEW: - Test Cases and Scripts have been added in accordance with the 2021D specifications release. - A 'Scan QR' button has been added to the QR-Code Data section in Project Properties > DUT Interview Data. - A new filter setting for the ZCP result upload has been added. It allows to select all form items where results are pending on ZCP. The Pending state is now displayed in the ZCP result column in the Form Item View (instead of keeping the column empty in such a case). - A button to the Project Properties > Configure Network windows has been added to the Project Start page and the main menu bar. CHANGED: - CTT 3.2 is now a Z-Wave Alliance tool. Installation, settings and document folders have been adjusted. Settings from older Silabs CTT versions are imported. - The default project folder has been moved from AppData to [User]/Documents/ (...). - If the DUT includes the CTT Controller in Project Properties > Configure Network, the CTT is set to SSC RT in order to not get the SIS role assigned. This ensures a non-SIS DUT will keep the Primary Controller network role. - The Command Classes XML file has been updated to version 2.9.0. The updated XML contains the fixed Command name for the Meter Table Capability Report in the Meter Table Command Class. - The Form Item View window is opened automatically when a Portal/ZCP-based project is loaded. - In the CTT UI, help pages and in most Test Case and Script log output inclusionary terms like 'End Device' is now used. Note that in specification text and some hardware related strings the terms have not yet been adjusted. - A Nonce Exchange is triggered whenever a Script run starts. This ensures that the recorded Zniffer trace can be decrypted. - Network keys are saved in the Script execution's log folder (same as for each executed Test Case). - Loading another project will reset/empty the Error Log window. - The search function in Wizard step 'Select Portal Case' works case- insensitive now. The filter function is now also applied if the Wizard is moved back to this step. - The warning messages when selecting a Portal case have been adjusted based on what permissions a user has and in what phase a Portal case currently is. - The link to the CTT update page that is displayed in the message log window when a CTT update is available now contains the correct Portal URL based on what Portal credentials have been used in CTT (Z-Wave Alliance Portal or Legacy Portal). - The Serial API firmware files from CTT Controllers and End Devices have been updated to SDK version 7.16.3. They are located in Users\[user]\ Documents\Z-Wave Alliance\Z-Wave CTT 3\CTT Device Firmwares. FIXED: - Scripts do not accept expected Commands on lower security schemes and these Commands will not change the security level anymore on which the script sends its own commands. - A potential memory leak when running the Firmware Update Meta Data Cmd Class scripts has been fixed. - When opening another DUT-based project the DUT Test Case Selection window is closed correctly. - Manual detection and verification of the Simplicity Commander path on the Options dialog has been fixed and improved. - Test Cases for supported Command Classes are now correctly assigned to the End Point nodes in the Test Case Explorer. - The Form-DUT Compare window is updated automatically after a Portal case re-sync. - A bug where Test Cases used older versions of the Command Classes XML file has been fixed. - A bug has been fixed where the CTT got into a blocked state after starting a Test Case without any selected hardware. - Double clicking on Script syntax or execution errors listed in the Error Log window will navigate to the script code position again. This function was broken in version 3.1.3. - The Load Default Layout function has been fixed. - The context menu entries have been fixed in Test Case Explorer. Sometimes wrong menu labels appeared after jumping between Test Case categories. - The References window is now always updated when another Project is loaded. CTT 3.1.3 (2021-03-26) ====================== NEW: - Created a Project Start Page with links and explanation to most important Project tools and views. - List with required Z-Wave hardware in Test Reporter added, including current state (available / missing). - List of missing Test Case hardware is shown in Property view at the bottom right of the CTT. - CTT allows to setup and use up to three Controller and three Slave hardware devices (used for additional S2 Test Cases). - Project Properties > Serial Devices dialog adjusts automatically to the number of required hardware devices (based on what Test Cases require). - CTT is able to use different Portal instance addresses when creating Portal-based projects, the list of available Portal addresses can be modified remotely. - Added Previous/Next Error buttons to Output Log view in Test Reporter. - Added path to Simplicity Commander to Options dialog so it can be selected manually if auto detection failed. - Added a new set of S2 certification Tests Cases (S2C_) for Controllers that are not based on Z/IP Gateway. - Added new set of Z-Wave Long Range specific Wiki page Test Cases (LR_). CHANGED: - DUT vs. Form Comparison: Advertised DUT Command Class names are shown even if the Command Class version is not known to CTT (not included in Command Class XML file). - Link to ‘Open Official Solution' has been removed from Start Page. - CTT menus and context menus have been cleaned up. - Script sequences are now also checked and executed when a Command Class script is checked. - Log window is switched to Output tab when any log output is added. Minimum size for log window is enforced. FIXED: - Security status of Basic Command Class in DUT is handled correctly now. - Fixed a crash when Portal and DUT End Points differed. - Fixed an exception when the CTT main Controller connection was established more than 127 times. - Project load the correct Command Classes XML file for script modifications. - Fixed a memory leak that happened if CTT received frames with Transport Encapsulation (e.g. during script execution). - A large number of fixes and improvements in Scripts and Test Cases. CTT 3.0.7 (2020-12-29) ====================== +---------------------------------------------------------------------------+ | Important notice when upgrading to version 3 | | '''''''''''''''''''''''''''''''''''''''''''' | | | | CTT v2 is obsoleted. CTT v3 has major changes compared to CTT v2. Please | | consult the 'Introduction' section of the help pages to learn how to use | | CTT v3. | | | | Solutions / projects from CTT v2 are NOT compatible with CTT v3. A new | | project has to be created in CTT v3 for a Device Under Test (DUT). | | | | CTT v3 is installed in parallel to any old CTT v2 installation. This | | means CTT v2 can still be used to load old CTT v2 solutions. Another way | | to use custom scripts from CTT v2 is to manually copy the source code of | | a CTT v2 script into a newly created CTT v3 script. | | | | If using the Z-Wave Plus v2 (ZWPv2) program, CTT v3 is able to download | | the set of required Self-Certification Test Cases from the Z-Wave | | Certification Portal (ZCP) and to upload Test Case results back to the | | ZCP. ZWPv2 form revision 5 or newer is required. | | | | If using the Z-Wave Plus (version 1) program, Test Cases are based on | | interview data of the DUT. This set of Test Cases may not be complete in | | regards to verifying Z-Wave certification requirements. | +---------------------------------------------------------------------------+ NEW: - First public release.