This paper reports on a case study of work analysis using geospatial intelligence technologies conducted on a manufacturing line at a J-Power Systems (JPS) plant in FY2020. First, an overview of the workplace, the purpose of the analysis, and the types of data used in the analysis is presented. Next, we describe the data processing methods developed in the preparatory phase of the analysis, and finally, we report the results and discussion of the work analysis.
Presented in APMS 2022
YouTube:
https://youtu.be/eFi8Z0T25Cc
Analyzing Operations on a Manufacturing Line using Geospatial Intelligence Technologies
1. Analyzing Operations on a
Manufacturing Line using
Geospatial Intelligence Technologies
Takeshi Kurata1
Munenori Harada2, Katsuko Nakahira2
Takashi Maehata3, Yoshinori Ito4, Hideki Aso4
1 Human Augmentation Research Center, AIST, Chiba, Japan
2 Faculty of Engineering, Nagaoka University of Technology, Nagano, Japan
3 IoT R&D Center, Sumitomo Electric Industries, Ltd. (SEI), Osaka, Japan
4 IoT Acceleration Lab, J-Power Systems Co. Ltd. (JPS), Ibaraki, Japan
2. Health and Productivity management (HPM):
Labor productivity and QoW
2
Kurata, T., Geospatial Intelligence for Health and Productivity Management in Japanese Restaurants and
Other Industries, APMS, pp. 206–214 (2021) doi: 10.1007/978-3-030-85906-0_23
3. HPM and Geospatial Intelligence (GSI)
3
Kurata, T., Geospatial Intelligence for Health and Productivity Management in Japanese Restaurants and
Other Industries, APMS, pp. 206–214 (2021) doi: 10.1007/978-3-030-85906-0_23
4. Case study of work analysis using indoor GSI techs
4
GSI: Geospatial Intelligence, GT: Ground-Truth, HS: Handwritten Stoppage factor,
PLC: Programmable Logic Controller, WIP: Work-In-Progress
@JPS (J-Power Systems Co. Ltd.)
About 1,800 m2 rectangle in size
5. Objectives of this work analysis
5
To investigate the feasibility of
(1) improving the operation rate
(2) Automating work records
(3) Visualizing work transitions
To confirm
(4) The value of work skills
QoW (Quality of Working)
(physical/mental load reduction, transparency
of work status, and skill improvement)
Productivity
6. Basics of data in this case study
6
Types of time-series work classification data
GT: Ground-Truth, ML: Manufacturing Line, LSTM: Long short-term memory
8. Methodology of work analysis on ML axis:
Quasi-GT (Ground-Truth) data generation
8
HS record: Handwritten Stoppage-factor record, ML: Manufacturing Line, MR: Material Replenishment
Method of Quasi-GT data generation
Pro: No missing data
Cons: Rounded-off recorded time
MR work not recorded
HS records
IoT transition: Still underway
Limited types of PLC data
PLC data
Positioning data
More accurate in time
MR work included
Quasi-GT data
MR detection process
Merging
process
9. Methodology of work analysis on ML axis:
Quasi-GT (Ground-Truth) data generation
9
ML: Manufacturing Line, MR: Material Replenishment, HS record: Handwritten Stoppage-factor record
MR detection process
10. Methodology of work analysis on ML axis:
Quasi-GT (Ground-Truth) data generation
10
HS record: Handwritten Stoppage-factor record
11. Work analysis on ML axis based on quasi-GT data
11
Quasi-GT data-based transition diagram (work process model) on the ML axis
12. Work analysis on ML axis based on quasi-GT data
12
Operation rate and continuous operation
time within each operation cycle
MR: Material Replenishment, HS record: Handwritten Stoppage-factor record
Time for MR and the approaching action
Continuous operation time: 60 sec
longer
⇒ Operation rate: 2% higher
Replace long approaching times
by the median
⇒ Operation rate: 3% higher
13. Work analysis on ML axis based on quasi-GT data
13
Operation rates in
passive and active cycles
MR: Material Replenishment
MR more active
⇒ Operation rate:
2% higher
15. Methodology of work analysis on ML axis:
Automatic work classification based on LSTM
15
Example of travel distance map from each
position to the target equipment for
position-based features
LSTM-based work classification
LSTM: Long short-term memory
16. Methodology of work analysis on ML axis:
Automatic work classification based on LSTM
16
Affected?
Issues for obtaining workers‘ status more:
Improve positioning accuracy where/when
PLC data is not available due to no
equipment controls for S*
Issues for improving the quality of S*
training data:
Avoid increasing the burden in the
workplace for collecting high-quality
training data
18. Methodology of work analysis on ML axis: Indoor positioning
18
Kurata, T., Sensing of Service Provision Processes, Service Engineering for Gastronomic Sciences, pp.67-85, 2020.
Potorti, F., et al., Off-line Evaluation of Indoor Positioning Systems in Different Scenarios: The Experiences from IPIN 2020 Competition, IEEE Sensors Journal (2021)
Indoor positioning tech map
xDR Challenge in IPIN
(Int’l indoor positioning
competitions )
19. Methodology of work analysis on ML axis: Indoor positioning
19
xDR: Dead Reckoning for X,PDR: Pedestrian Dead Reckoning, VDR: Vehicle Dead Reckoning, BLE (Bluetooth Low Energy)
Kurata, T., et al., IoH Technologies into Indoor Manufacturing Sites, APMS, pp.372-380 (2019)
The shortest transmission interval:
1.28 sec.
(The darker, the longer)
Install battery-powered beacons at critical areas where PLC data is not available
due to no equipment controls for setup-related work classes S*
CE50 of JPS/SEI system: 4.39 m
(cf. CE50 of AIST system: 2.73 m)
20. Work analysis on ML and worker axes based on GT data
20
WA (WIP output Adjustment) , MWR (Manual Work Record), and
CL (Cleaning and garbage disposal) in OP (Operation)
Notes de l'éditeur
Health and productivity management (HPM) is realized by simultaneously improving labor productivity and Quality of Working (QoW) in a well-balanced manner. Since it is necessary to deal with a wide range of issues like this, engineering and DX approaches are essential.
On the other hand, it has been reported that 60-80% of information is related to location and that humans spend approximately 90% of their time indoors. So, geospatial intelligence (GSI), especially indoor GSI, which supports problem solving by linking geospatial data with other data, is expected to be an effective means to promote DX and HPM.
In this presentation, I am going to report on a case study of work analysis using indoor GSI technologies conducted on manufacturing lines (MLs) at a JPS plant. An indoor positioning system and online PLCs were installed to collect data
(to determine the position of each worker and the state of each equipment. There are three MLs with three workers in charge of one line each and a leader of the MLs manages the entire work area.)
The main objective of this work analysis was to investigate the feasibility of (1) improving the operation rate, (2) automating work records and (3) visualizing work transitions, respectively, and to confirm (4) the value of work skills. From the perspective of HPM, (1) to (4) are all related to productivity, and (2) to (4) are related to QoW such as physical/mental load reduction, transparency of work status, and skill improvement.
This slide shows four types of data in this case study. The "ground-truth (GT) data" were prepared manually by visual inspection. Only the GT data have both the ML axis and the worker axis. The handwritten stoppage-factor records "HS records" are daily handwritten records kept by workers in the workplace. The "quasi-GT data" are generated by integrating the HS records and PLC data using the method explained later. Since both HS records and PLC data are constantly obtained onsite, they can be easily scaled up. The "auto-estimated data" are the estimation results of a method based on machine learning LSTM using PLC data and positioning data. All three types of data are only along the ML axis.
This slide shows the breakdown of work classifications such as MR, OP, setup (S) series,
(as well as the GT values and work shift overlaps for workers #1, #2, and #3.)
The HS records contain no missing data (along the ML axis). However, there are two significant issues. One is that the start and end times of each record are rounded off, and the other is that "material replenishment (MR)" works are not recorded even if MR is one of major stoppage factors of MLs. Meanwhile, because IoT transition is still underway in the workplace, PLC data that can be sent to the server and used for analysis are limited So, in this case study, we developed a method to generate "quasi-GT data" that are more accurate in time and include MR works. I will show you this MR detection process with the next slide.
BTW, the MR detection process is also used to extract one of PLC-based features for LSTM. With PLC data, it is possible to grasp each status of "out of material" and "open/closed status of the equipment cover“, so their combinations are used in the detection process like this.
The accuracy of the quasi-GT data generation method was evaluated by comparing it with the HS records and GT data. As a result, the quasi-GT data can be generated with an accuracy of 92%. This is better than HS records, and we can automatically record operation (OP) and MR with 96% and 93% respectively.
We conducted work analysis using the quasi-GT data and GT data instead of using the results of machine learning since it was still in the feasibility study phase. First here is about the quasi-GT data-based analysis on the ML axis. This figure shows a state transition diagram of the work classification. The transitions from OP clearly identify that the major stoppage factor is MR and that the main internal setup workflow after OP is "SC to {SA or SD} to SE." Actually, this is a work process model, so it is possible to perform simulations based on this, and it are expected for efficient ‘Kaizen’ improvement.
As shown in the left graph, there was a weak positive correlation between the operation rate and the continuous operation time within each operation cycle. These results implies that the longer the continuous operation time is, the higher the operation rate is. As each ML requires a brief stop for MR, so we analyzed MR and the approaching actions to the equipment for MR. Comparing the HS records with the quasi-GT data confirmed that 34% of time in OP of the HS records were actually MR. Also, it was found that the approaching time obtained by the MR work detection process accounted for 36% of the MR time. (If all approaching times that take longer than the median were replaced by the median, the operation rate could be improved by 3%.)
We also compared passive and active MR cycles. In a passive cycle, all approaching actions are carried out after the material runs out, and in an active cycle, some of approaching actions started before the material runs out. The analysis confirmed that the median operation rates for passive and active cycles are 47% and 49%, respectively, (and that the variation in operation rates is greater for passive cycles.)
Here is the summary of this case study. I skipped some of other results, but we quantitatively found out that work skills can influence the productivity, and we expect that being able to achieve higher productivity through skill improvement and such visualization will make the work more rewarding.
For the rest of this presentation, I briefly introduce automatic work classification based on LSTM to achieve automatic recording without HS records. This LTSM uses 47 features derived from PLC data and 9 features derived from positioning data. As for features based on positioning data, we designed travel distance features. For efficient computation, maps of travel distances from each position to the target equipment are pre-computed as in this figure. (In addition, the time spent in each work area is also used as other position-based features.)
The purpose of work classification estimation was to automatically generate data comparable to the quasi-GT data, which contributes to elimination of manual recording works, so the comparison target here was the quasi-GT data. As a result of evaluation by k-fold cross-validation in which K=3, the estimation accuracy was about 91%. While OP and MR are highly accurate, there was room for improvement for estimating setup-related work classes, S*. Especially, to estimate them involving no equipment operations, it is crucial to obtain the workers' status because, in such situation, PLC data is not available. So, positioning data is expected to play more important role.
This is the indoor positioning technology map.
The integrated positioning method used in this case study consists of BLE (Bluetooth Low Energy) positioning, PDR (Pedestrian Dead Reckoning), and map matching. Since the data for international indoor positioning competitions named xDR Challenge were collected at the same workplace as this case study, here we used the competition data for performance evaluation.
As a result, CE50, a representative accuracy indicator, was 4.39 m (Reference: CE50 in AIST's method was 2.73 m).
At the target workplace, solar BLE beacons (Fujitsu PulsarGum) were installed for the maintenance-free advantage. There are many areas where the luminance level is not sufficient such as aisles and other areas. In such environments, the advertisement transmission interval is getting longer.
Therefore, it would be possible to improve the positioning accuracy and the quality of features related to setup works by installing battery-powered beacons at critical areas where PLC data is not available for work classification.
here is about the GT data-based analysis on the ML and worker axes. The findings are like:
"WIP output adjustment (WA)" (mainly position adjustment of WIP output in the container) occupies about half of the time in OP,
the ratio of "manual work record (MWR)" in OP to the one except in OP is 2:1,
and about half of "cleaning and garbage disposal (CL)" is done during internal setup.
For example, if workers for Shifts #1, #2, #3, and #5 works similarly to the high-skilled worker in Shift #4, WA can be replaced with other works such as MWR and CL, and the ones in OP can be increased by 8% of time for OP. Although this 9% is only an average, this 8% is expected to be within the 9%.