DeviQA and Soapbox: WMS functional testing case analysed

Verdict: what is checkable

The evidence grade (this site's label for how a case is sourced: primary with client or press confirmation, secondary when vendor-hosted only.) is secondary: the only source is the vendor's own page; of 18 metrics, 0 are verifiable yes, 18 partial and 0 no. [ev-0891]

Client profile as published

Client
Soapbox
Segment
Ecommerce and D2C (direct to consumer: a brand that sells its own products online to shoppers without a retailer in between.)
Region
not stated in the source
Channels
ecommerce
Size indicators
none given

The client description and size indicators are quoted from the source; segment, region and channels are this site's classification. [ev-0891]

Systems in scope

  • WMS (warehouse management system: software that runs receiving, storage, picking and shipping inside a distribution center.): named in the title and the About project section ("e-commerce fulfillment and warehouse management platform")
  • OMS (order management system: software that routes, splits and tracks customer orders across stores, warehouses and carriers.): named in the About project section ("order processing")

Where the source names each system is quoted or summarised from the case page. [ev-0891]

Problem as stated

Before the engagement Soapbox ran tests by hand, test coverage stood at 40%, a regression run took about 1.5 weeks and stories took about two weeks to move forward. The page describes the QA process as poor and the Android app code as carrying many major and critical issues. Soapbox asked DeviQA to structure test documentation, widen coverage and cut testing time on the web platform and the Android app prototype. [ev-0891]

A huge amount of major/critical issues in the code base of the mobile app (quoted from the source)

Approach

Testing types
functional, automation and mobile (main scope: functional)
Team shape
3 manual QA engineers, 1 automation QA engineer, 1 mobile developer
Duration
Since 2021
Engagement model
dedicated-team
Tooling named in the source
Figma, GitLab, GitLab issues, Sentry, Datadog, Android Studio, Cypress, MS Visual Studio Code, Google Docs, Google Sheets, React Native, React Native Reanimated, React Navigation, App Center, Swagger

Metrics claimed

Metrics as published, with who claims each and what this site could check.
MetricValueUnitClaimed byVerifiableEvidence
test coverage before DeviQA40%vendorpartialev-0892
software covered with test cases>90%vendorpartialev-0893
test cases and automated scripts created2,500+test cases and automated scriptsvendorpartialev-0894
web platform coverage90%vendorpartialev-0894
test cases created (stat block)2.5k+test casesvendorpartialev-0895
automated scripts in the first 6 months>400automated scriptsvendorpartialev-0896
regression run time1.5 weeks to three daysvendorpartialev-0897
regression testing speed (stat block)X3xvendorpartialev-0898
time to move stories forward before DeviQA~2weeksvendorpartialev-0899
task handling speed2xxvendorpartialev-0900
share of code refactored60%vendorpartialev-0901
mobile app files refactored200+filesvendorpartialev-0902
successful releases95+releasesvendorpartialev-0903
web platform releases supported80+releasesvendorpartialev-0904
Android app releases supported15+releasesvendorpartialev-0904
critical bugs reported550+bugsvendorpartialev-0905
defects reported1,500+defectsvendorpartialev-0906
share of defects blocker, critical or major60%vendorpartialev-0906

What the case does not say

  • Measurement method.
  • Duration over which the metric held.
  • Cost.
  • What was out of scope.

Reading notes for a buyer

This case speaks to the scenario “OMS or WMS replacement” and feeds dimension D2 (Retail system coverage) of the methodology. A question to put to DeviQA:

Which warehouse and order flows did the web coverage figure include, and how was coverage counted? [ev-0891]

Source and verification

Source
QA services for warehouse management software | DeviQA case study
Published
not dated by the vendor
Verified
2026-10-30
Evidence grade
secondary
Corroborating sources
none found
Evidence records for this case. First 12 of 16 shown; the rest open below.
IdClaimSourceClassGradePublishedVerified
ev-0891QA services for warehouse management softwareQA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-089240% test coverageQA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0893>90% of the software is covered with detailed and relevant test casesQA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0894DeviQA created 2,500+ test cases and automated scripts, achieving 90% coverage of the Soapbox web platform.QA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-08952.5k+ Test cases createdQA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0896>400 automated scripts were created in the first 6 monthsQA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0897Regression testing decreased from 1.5 weeks to three days.QA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0898X3 Faster regression testingQA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0899~2 weeks to move stories forwardQA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-09002x faster task handlingQA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-090160% of the code was refactored to improve application stabilityQA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0902200+ Refactored files of the mobile appQA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
Show 4 more evidence records
Remaining evidence records.
IdClaimSourceClassGradePublishedVerified
ev-090395+ successful releasesQA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0904DeviQA supported 80+ releases of the Soapbox web platform and 15+ releases of its Android application.QA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0905550+ Critical bugs reportedQA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0906The team reported 1,500+ defects, with 60% classified as blocker, critical, or major.QA services for warehouse management software | DeviQA case studyvendor-case-studysecondary2026-10-30

Vendor profile: DeviQA. System facet: OMS and WMS testing vendors.

Positions on this page are computed from the methodology published at /methodology/ and from the evidence records linked on this page. Case study figures are the vendor's or the client's claims as published at the linked source; the "verifiable" column states what this site could check. The site does not recommend vendors, does not accept paid placement and does not publish rates. Last computed: 2026-11-03.

Published by the editor of retailqavendors.com. Last reviewed .