Digital Gauge FAQ

Indication accuracy of linear gauge

1. Indication accuracy

■ Indication accuracy

Indication accuracy is the measurement error of the linear gauge sensor. An error is measured for each specified measurement value and the sum of the absolute values of the maximum error in the plus direction and the maximum error in the minus direction in terms of the total spindle moving length is defined as the indication accuracy of the sensor.

• • Indication accuracy is 3 µm even in the minimum unit of 10 µm

Adjust the 0 of gauge sensor and reference device to start the measurement. For example, when using the GS-102, read the value of gauge sensor itself and the reference device at the values of 2.01, 4.02, 6.03, 8.04 and 10.05 mm. The difference of the value is an error.
Even with the gauge sensor with a resolution of 10 µm, the indication accuracy 3 µm (catalog value) is obtained because the minimum reading value of the reference device is smaller than 10 µm.
Also, the reason that the value to be measured is not an integer is to confirm the multiplication accuracy together.

■ Measurement example

Measurement point
(mm)
Reading of reference device (mm) Error (µm)

0

0

0

2.01

2.0095

0.5

Max

4.02

4.021

-1

6.03

6.0322

-2.2

Min

8.04

8.0413

-1.3

10.05

10.052

-2

Indication accuracy= Max − Min = 0.5 −(−2.2)= 2.7 µm

• ■ Resolution

The resolution is the minimum reading value that can be read with the combination of the gauge sensor and display unit. For example, 10 µm (0.01 mm) is the minimum reading value for the GS-102.

• ■Multiplication accuracy

When the resolution and scale division of the internal sensor is the same, the accuracy for the scale becomes the accuracy for the resolution. In general, the resolution is obtained by electrical equal division of the scale. An error caused by this division is called multiplication accuracy.

2. Supplement

When it is GS-551 (for minimum unit 1 µm), the signal of 1 pulse per 1 µm is created by the electrical processing of the slit with pitch of 8 µm.
The accuracy for every 8 µm corresponding to the point of the slit is shown in the <Table 1>.
Electric processing error is generated between slits. Those values are displayed in <Table 2>. For checking the error of electrical processing, the fraction is omitted.
1 count (1 pulse=µm) error in the display side is added because of the pulse signal.
Slit interval is varied depending on the model.

<Table 1> 2 places measurement data in 5 mm stroke

Measurement point
mm
Error 1
(1 to 3 mm)
µm
Error 2
(1.7 to 3.7 mm)
µm
0.2 0.0 0.1
0.4 0.0 0.0
0.6 0.0 0.0
0.8 0.0 −0.1
1.0 0.1 0.0
1.2 0.1 0.1
1.4 0.2 0.0
1.6 0.1 0.1
1.8 0.2 0.0
2.0 0.2 0.0

<Table 2> Measurement data of 2 units

Measurement point
mm
No.1 error
µm
No.2 error
µm
1.002 −0.8 1.2
2.004 −0.6 1.4
3.006 −0.6 0.8
4.008 −0.7 0.2
5.010 −1.1 1.2
Accuracy(Max. value) 1.1 1.4

Sensor error is combination of Mechanical error (slit pasting error, bents of spindle, clattering, etc.) and Electrical error (multiplication error)

* The slit pitch for the GS-551 is 8 µm (=0.008 mm), so as long as accuracy measurement is performed at the intervals of this multiple, multiplication error do not occur. This is why the error is small.

Revised:2002.03.15