Skip to main content
Skip table of contents

Appendix B - Noise and Resolution Tables [U6 Datasheet]

A Note About ADC Noise and Resolution

Analog voltages measured by the U6 are converted to digital representation via the U6's analog to digital converter (ADC).  The ADC reports an analog voltage in terms of ADC counts, where a single ADC count is the smallest change in voltage that will affect the reported ADC value.  A single ADC count is also known as the converter's LSB voltage.  The ADC's resolution defines the number of discrete voltages represented over a given input range.  For example, a 16-bit ADC with a +/-10 input range can report 65536 discrete voltages (216) and has an LSB voltage of 0.305 mV (20V รท 216).

The stated resolution for an ADC is a best case value, assuming no channel noise.  In reality, every ADC works in conjunction with external circuitry (amplifiers, filters, ect.) which all posses some level of inherent noise.  The noise of supporting hardware, in addition to noise of the ADC itself, all contribute to the channel resolution.  In general, the resolution for an ADC and supporting hardware will be less than what is stated for the ADC.  The combined resolution for an in-system ADC is termed effective resolution (aka ENOB).  Simply put, the effective resolution is the equivalent resolution where analog voltages less than LSB voltage are no longer differentiable from the inherent hardware noise. In addition to defining the smallest measurable analog voltage,  the effective resolution also defines the RMS peak-to-peak noise on a given analog channel.

Closely related to the effective resolution is the error free code resolution (EFCR) or flicker-free code resolution.  The EFCR represents the resolution on a channel immune to "bounce" or "flicker" from the inherent system noise.  The EFCR is not reported in this appendix. However, it may be closely approximated by the following equation:

EFCR = effective resolution - 2.7 bits[1.]

The U6 offers user-selectable resolution through the resolution index parameter on any one AIN channel.  Internally, the ADC hardware uses modified sampling methods to increase measurement resolution beyond the ADC's base resolution.  Valid resolution index values are 0-8 for the U6 and 0-12 for the U6-Pro [2.][3.].  Increasing the resolution index value will improve the channel resolution, but doing so will usually extend channel sampling times.

Noise and Resolution Data

The data shown below summarizes typical effective resolutions and expected channel sampling times over all resolution index values.  Data for the U6 and U6-Pro data are combined and presented together for convenience, where resolution index values 9-12 only apply to the U6-Pro.

The AIN sampling time is the typical amount of time required for the ADC hardware to make a single analog to digital conversion on any channel and is reported in milliseconds per sample.  The AIN sampling time does not include command/response and overhead time associated with the host computer/application.

Noise and Resolution Test procedure

Noise and resolution data was generated by collecting 512 successive voltage readings, using a short jumper between the test channel and ground.  To get the effective resolution in volts, we simply take the standard deviation of this array of voltage readings:

Effective Resolution in Volts = StandardDeviation (Data Array in Volts)

To calculate effective resolution in bits, we first convert the voltage readings to 16-bit aligned values.  Essentially 16-bit binary values but with decimal places.  We then take the standard deviation of those values, and then use the last equation below to calculate effective resolution in bits:

16-bit Aligned Value = 65536.0 * ((Voltage - MinSpanVolts) / (MaxSpanVolts - MinSpanVolts))

RMS Noise = StandardDeviation (16-bit Aligned Values)

Effective Resolution in Bits = 16.0 - log2 (RMS Noise) 

See Appendix A for the min and max span voltages.  For example, with Gain = 1 (Range = 10), the min is about -10.6 volts and the max is about 10.1 volts.

Estimating Command/Response Times

Effective resolution and sampling times for various gains and resolution index settings. Resolution index settings 9-12 apply to the U6-Pro only.


1.  The equation used to approximate the EFCR is determined using +/-3.3 standard deviations from the RMS noise measured on an AIN channel.

2.  Resolution index 0 defaults the U6 to resolution index = 8 and the U6-Pro to resolution index = 9 in command response mode.  Stream mode does not support the 24-bit ADC.  Therefore, setting the resolution index to 0 is equivalent to resolution index = 1.

3.  The U6-Pro is equipped with a 24-bit delta-sigma ADC, in addition to the standard 16-bit ADC.  Analog conversions occur on the 16-bit ADC when resolution index values 0-8 are used.  Analog conversion occur on the 24-bit ADC when resolution index values 9-12 are used (command response mode only).

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.