For a digital indicator with 0.001 mm resolution, do I still add a resolution component if I already have repeatability from 10 readings…
Global Search
Search Metrology Atlas
One query across documents (page by page), definitions, standards, questions, news, tools and training.
28 results for “Resolution”
Dictionary
Resolution
…Digital resolution contributes a rectangular uncertainty component of (δ/2)/√3. VIM (JCGM 200:2012) 4.14 Instruments
Dictionary
Resolution of a displaying device
The smallest difference between displayed indications that can be meaningfully distinguished — for a digital display, one count of the least significant digit. VIM (JCGM…
…A longer time interval for the calibration may be required, especially with timing devices with larger resolutions. Report the calculated uncertainty to two significant…
Documents
SOP 5 — 3 1 Weighing Design
…Calibrated barometer with sufficiently small resolution, stability, and uncertainty (See NISTIR 6969, SOP 2, e.g., accurate to ± 66.5 Pa (0.5 mmHg…
Documents
SOP 7 — Single Substitution
…Apparatus/Equipment Required Precision analytical balance or mass comparator with sufficient capacity and resolution for the calibrations planned. Calibrated working standards, of nominally equal…
…Barometer with resolution and uncertainty less than 135 Pa (1 mmHg to determine barometric pressure (absolute). Hygrometer with resolution and uncertainty less than 10…
Documents
SOP 14 — Gravimetric Calibration
…The sensitivity or resolution of the balance may be a limiting factor in the accuracy of the measurement. The resolution and repeatability must be…
Documents
SOP 49 — Cal Env Stds
…less than resolution up Estimated process uncertainty based on resolution of the instrument being calibrated Rectangular or rounded up to nearest resolution Estimates of…
Documents
SOP 4 — Double Substitution
…https://doi.org/10.6028/NIST.IR.6969 - 2019 Calibrated barometer with sufficiently small resolution, stability, and uncertainty (See SOP 2, e.g., accurate…