ScienceSoft and an anonymised omnichannel retailer: Magento functional testing case analysed
Verdict: what is checkable
The evidence grade (this site's label for how a case is sourced: primary with client or press confirmation, secondary when vendor-hosted only.) is secondary: the only source is the vendor's own page; of 9 metrics, 0 are verifiable yes, 0 partial and 9 no. [ev-1698]
Client profile as published
- Client
a US-based multi-business corporation present in a variety of industries, including hospitality, retail (ecommerce and brick-and-mortar fashion stores), and wellness
(the vendor's description; the vendor does not name the client)- Segment
- Enterprise omnichannel
- Region
- United States
- Channels
- stores and ecommerce
- Size indicators
multi-business corporation present in a variety of industries
The client description and size indicators are quoted from the source; segment, region and channels are this site's classification. [ev-1698]
Systems in scope
- Ecommerce platform (Magento): named in the Challenge section (Magento-based ecommerce platform) and in the technologies list (Magento 2)
- Integrations: named in the Solution section ("the platform’s integrations with such external systems as CMS, a PIM tool, a website payment and a POS (point of sale: the till software and hardware that rings up store sales, takes card payments and prints receipts.) system, a warehouse management tool, CRM, and a tax calculator")
- POS: named in the Solution section as an integration of the ecommerce platform and as a source of customer data checked in CRM and DWH testing
- Payments: named in the Solution section (website payment integration; "payment-transaction processes complied with PCI DSS (Payment Card Industry Data Security Standard: the security rules for any business that stores, processes or transmits card data.)")
Where the source names each system is quoted or summarised from the case page. [ev-1698]
Problem as stated
The client's Magento store, a Dynamics 365 CRM for web and mobile, and an on-premises data warehouse were all still being built and customized. The client wanted one managed testing provider with Magento, Dynamics and data warehouse experience to check function, security and usability, and to confirm that new changes did not break existing features. [ev-1698]
Approach
- Testing types
- functional, performance, security and mobile (main scope: functional)
- Team shape
- six test engineers on the ecommerce platform, two test engineers on web and mobile CRM testing, one test engineer on the data warehouse; team size scaled up and down
- Duration
- not stated
- Engagement model
- managed-testing
- Tooling named in the source
- Magento Test Framework, Apache JMeter, Atlassian JIRA, Atlassian Confluence
Metrics claimed
| Metric | Value | Unit | Claimed by | Verifiable | Evidence |
|---|---|---|---|---|---|
| test engineers on the ecommerce platform | six | test engineers | vendor | no | ev-1699 |
| test engineers on CRM testing | two | test engineers | vendor | no | ev-1700 |
| user stories and test casesvendor claim, outside the typical range in this data set | 2,200 | user stories and test cases | vendor | no | ev-1701 |
| access levels for brick-and-mortar stores | 8 | access levels | vendor | no | ev-1702 |
| access levels for hospitality employees | 7 | access levels | vendor | no | ev-1702 |
| access levels for wellness centers | 5 | access levels | vendor | no | ev-1702 |
| iOS devices in mobile testing | three | iOS devices | vendor | no | ev-1703 |
| Android devices in mobile testing | two | Android devices | vendor | no | ev-1703 |
| client records in DWH load check | 5,000+ | real client records | vendor | no | ev-1704 |
What the case does not say
- Baseline before the engagement.
- Measurement method.
- Duration over which the metric held.
- Cost.
- What was out of scope.
- Whether the client is still a client.
- Date of the work.
Reading notes for a buyer
This case speaks to the scenario “Payments and PCI scope” and feeds dimension D2 (Retail system coverage) of the methodology. A question to put to ScienceSoft:
Which POS and payment integration scenarios did the team test end to end, and what did the PCI DSS check cover? [ev-1698]
Source and verification
- Source
- Managed Testing of an Ecommerce Solution, CRM and Data Warehouse for US Multi-Industry Corporation
- Published
- not dated by the vendor
- Verified
- 2026-10-30
- Evidence grade
- secondary
- Corroborating sources
- none found
| Id | Claim | Source | Class | Grade | Published | Verified |
|---|---|---|---|---|---|---|
| ev-1698 | Managed Testing of an Ecommerce Solution, CRM and Data Warehouse for US Multi-Industry Corporation | Managed Testing of an Ecommerce Solution, CRM and Data Warehouse for US Multi-Industry Corporation | vendor-case-study | secondary | 2026-10-30 | |
| ev-1699 | A group of six test engineers got down to testing as soon as the development process of the e-commerce platform started. | Managed Testing of an Ecommerce Solution, CRM and Data Warehouse for US Multi-Industry Corporation | vendor-case-study | secondary | 2026-10-30 | |
| ev-1700 | Thus, two ScienceSoft’s test engineers paid special attention to functionality, performance, usability testing and role-based access control validation during web and mobile CRM testing. | Managed Testing of an Ecommerce Solution, CRM and Data Warehouse for US Multi-Industry Corporation | vendor-case-study | secondary | 2026-10-30 | |
| ev-1701 | With a significant number of different user stories and test cases (2,200) to implement for retail, hospitality and wellness employees, functionality testing took the majority of the testing time. | Managed Testing of an Ecommerce Solution, CRM and Data Warehouse for US Multi-Industry Corporation | vendor-case-study | secondary | 2026-10-30 | |
| ev-1702 | To manage this process, the test team created a roles matrix: for brick-and-mortar stores, 8 access levels were assigned, for hospitality employees, 7 levels, and for wellness centers, 5. | Managed Testing of an Ecommerce Solution, CRM and Data Warehouse for US Multi-Industry Corporation | vendor-case-study | secondary | 2026-10-30 | |
| ev-1703 | The mobile app was tested on three iOS devices of different versions and two Android devices. | Managed Testing of an Ecommerce Solution, CRM and Data Warehouse for US Multi-Industry Corporation | vendor-case-study | secondary | 2026-10-30 | |
| ev-1704 | The test team also checked the DWH load limits, as it had to cope with the large volume of data (5,000+ real client records) and the maximum number of users that access the DWH through the e-commerce platform and CRM simultaneously. | Managed Testing of an Ecommerce Solution, CRM and Data Warehouse for US Multi-Industry Corporation | vendor-case-study | secondary | 2026-10-30 |
Vendor profile: ScienceSoft. System facet: Ecommerce platform testing vendors.
Positions on this page are computed from the methodology published at /methodology/ and from the evidence records linked on this page. Case study figures are the vendor's or the client's claims as published at the linked source; the "verifiable" column states what this site could check. The site does not recommend vendors, does not accept paid placement and does not publish rates. Last computed: 2026-11-03.
Published by the editor of retailqavendors.com. Last reviewed .