DeviQA and an anonymised specialty retailer: OMS functional testing case analysed
Verdict: what is checkable
The evidence grade (this site's label for how a case is sourced: primary with client or press confirmation, secondary when vendor-hosted only.) is secondary: the only source is the vendor's own page; of 14 metrics, 0 are verifiable yes, 0 partial and 14 no. [ev-0921]
Client profile as published
- Client
A New York art gallery
(the vendor's description; the vendor does not name the client)- Segment
- Specialty retail
- Region
- United States
- Channels
- not stated
- Size indicators
- none given
The client description and size indicators are quoted from the source; segment, region and channels are this site's classification. [ev-0921]
Systems in scope
- OMS (order management system: software that routes, splits and tracks customer orders across stores, warehouses and carriers.): named in the client description ("From managing orders and private sales to public exhibitions")
Where the source names each system is quoted or summarised from the case page. [ev-0921]
Problem as stated
The gallery runs a digital platform for managing, presenting and selling artworks, books and art objects, including orders and private sales. QA ran apart from the development and release flow, and testing happened mostly at the UAT stage. User stories had no structure, bug reports lacked context and many test cases were out of date. DeviQA was asked to tie QA into the release cycle, cut testing time and improve issue tracking. [ev-0921]
The QA process ran in isolation from the development and release flow(quoted from the source)
Approach
- Testing types
- functional (main scope: functional)
- Team shape
- 1 manual QA engineer
- Duration
- Since 2023
- Engagement model
- dedicated-team
- Tooling named in the source
- Jira, Confluence, GitHub, Slack, 1Password, Mailtrap, Google Docs, Postman, Microsoft D365
Metrics claimed
| Metric | Value | Unit | Claimed by | Verifiable | Evidence |
|---|---|---|---|---|---|
| release velocity | 30 | % | vendor | no | ev-0922 |
| critical production bugs | 25 | % | vendor | no | ev-0923 |
| overall testing time | 40 | % | vendor | no | ev-0924 |
| early-stage defect detection | 60 | % | vendor | no | ev-0925 |
| lost or unclear tickets | 95 | % | vendor | no | ev-0926 |
| task processing speed | 20 | % | vendor | no | ev-0927 |
| bug reports with screenshots or videos | 100 | % | vendor | no | ev-0928 |
| ticket reopen rate | 50 | % | vendor | no | ev-0929 |
| defect reproduction time | 30 | % | vendor | no | ev-0930 |
| regression risk | 70 | % | vendor | no | ev-0931 |
| test execution time | 35 | % | vendor | no | ev-0932 |
| test cases created | 515 | test cases | vendor | no | ev-0933 |
| test coverage | 90 | % | vendor | no | ev-0934 |
| bugs logged | 1k+ | bugs | vendor | no | ev-0935 |
What the case does not say
- Baseline before the engagement.
- Measurement method.
- Duration over which the metric held.
- Cost.
- What was out of scope.
Reading notes for a buyer
This case speaks to the scenario “First outsourced QA team for a mid-size retailer” and feeds dimension D1 (Retail case study evidence) of the methodology. A question to put to DeviQA:
How were the percentage changes measured, over what period, and what were the starting values? [ev-0921]
Source and verification
- Source
- QA for an art management and sales platform | DeviQA case study
- Published
- not dated by the vendor
- Verified
- 2026-10-30
- Evidence grade
- secondary
- Corroborating sources
- none found
| Id | Claim | Source | Class | Grade | Published | Verified |
|---|---|---|---|---|---|---|
| ev-0921 | QA for an art management and sales platform | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0922 | Release velocity increased by 30% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0923 | Critical production bugs decreased by 25% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0924 | Overall testing time decreased by 40% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0925 | Early-stage defect detection rose by 60% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0926 | Lost/unclear tickets decreased by 95% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0927 | Task processing speed increased by 20% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0928 | 100% of reports include screenshots or videos | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0929 | Ticket reopen rate decreased by 50% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0930 | Defect reproduction time decreased by 30% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0931 | Regression risk decreased by 70% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0932 | Test execution time decreased by 35% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0933 | 515 Test cases created | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0934 | 90% Test coverage | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0935 | 1k+ Number of bugs logged | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 |
| Id | Claim | Source | Class | Grade | Published | Verified |
|---|---|---|---|---|---|---|
| ev-0921 | QA for an art management and sales platform | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0922 | Release velocity increased by 30% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0923 | Critical production bugs decreased by 25% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0924 | Overall testing time decreased by 40% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0925 | Early-stage defect detection rose by 60% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0926 | Lost/unclear tickets decreased by 95% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0927 | Task processing speed increased by 20% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0928 | 100% of reports include screenshots or videos | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0929 | Ticket reopen rate decreased by 50% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0930 | Defect reproduction time decreased by 30% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0931 | Regression risk decreased by 70% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0932 | Test execution time decreased by 35% | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 |
Show 3 more evidence records
| Id | Claim | Source | Class | Grade | Published | Verified |
|---|---|---|---|---|---|---|
| ev-0933 | 515 Test cases created | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0934 | 90% Test coverage | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0935 | 1k+ Number of bugs logged | QA for an art management and sales platform | DeviQA case study | vendor-case-study | secondary | 2026-10-30 |
Vendor profile: DeviQA. System facet: OMS and WMS testing vendors.
Positions on this page are computed from the methodology published at /methodology/ and from the evidence records linked on this page. Case study figures are the vendor's or the client's claims as published at the linked source; the "verifiable" column states what this site could check. The site does not recommend vendors, does not accept paid placement and does not publish rates. Last computed: 2026-11-03.
Published by the editor of retailqavendors.com. Last reviewed .