Antonio Aloisi European University Institute, Florence Elena Gramano Goethe University, Frankfurt ILO, Geneva, 28 Feb. - 1 Mar. 2019

1. Artificial Intelligence
is Watching You at Work
Digital Surveillance, Employee
Monitoring & Regulatory Issues
from a Civil Law Perspective
Antonio Aloisi European University Institute, Florence
Elena Gramano Goethe University, Frankfurt
ILO, Geneva, 28 Feb. - 1 Mar. 2019

2. outline of today’s presentation
AI & control
power
2
GDPR &
automated
processes
National
cases
(FR, DE, IT)
Final
remarks
1 2 3 4

3. big data analytics and algorithmic governance
POWER RELATIONSHIPS
New technologies are reshaping the
world of work in an ever-growing number
of fields. They have been redesigning
workplace interactions and power
relationships since ever, but the full
potential of some digital instruments has
not been unleashed in terms of pace,
scale and scope
AI BOOM
The wave of industrial development is
acclaimed as the “second machine age”,
boosted by the proliferation of
cyber-physical infrastructure and
interconnected systems making possible
practices of surveillance and profiling,
the resulting gigantic datasets lay the
groundwork for the AI boom
Powerful and multifarious innovations which, when it comes to recruiting,
organizing, monitoring and disciplining the workforce, can be considered as
an effective combination of big data analytics and algorithmic governance.
3
>

4. “instead of facilitating an emancipating new
environment, intrusive technology are
used to deepen hierarchy and control
over work performance, team dynamics,
usage habits, social behavior and even
biometric characteristics

5. Only recently, European and
domestic institutions started
considering how to update
existing regulation in order to face
the complex and far-reaching
challenges posed by tech devices
and, more specifically, by artificial
intelligence, a general-purpose
application able to mimic adaptive
and predictive “functions that
humans associate with their own
intelligence”
5

6. AI & control power:
a preliminary
conceptualization
6

7. direct interaction with new generation machineries
▸ increasingly blurred boundaries between professional and private lives,
▸ cheap, massive and less visible production, capturing, collection and
usage of data,
▸ effective cloud storage and computing, machine learning, Internet of
Things (IoT), neuronal networks and mobile robotics...
represents the lifeblood of the current remodeling, “creating significant
challenges to privacy and data protection” [WP29, Opinion 2/2017 on data
processing at work, WP 249, 8 June 2017]
today almost every occupation, standard or non-standard and
across different contexts (NOT only gig work!), involves a direct
interaction with tech devices, or even a complete reliance on them.
7

8. beyond the workless future
It is not the number of jobs lost
through advanced automation
(artificial artificial intelligence)
which should worry public opinion
most but rather the subtle potential
of AI and algorithms, leading to a
model of control and appraisal
without an intuitive link between
what is done when “logged-in” and
how it is assessed.
8

9. the ILO’s view
process
management
tasks
matching tasks
ride-hailing services,
accommodation
services, retail or human
resource management
classification tasks
9
“reducing costs on finding
customers or suppliers and
offering less expensive
solutions to their growing
customer base”
“integrate supply chains
through better information
about product quality,
certification schemes and
market conditions”
“recognition techniques in
relation to the increase in
surveillance”

10. AI’s functions
scan and hire
control and
discipline
10
data, data, data
for what else?
>
assess
>

11. Janus-faced
While AI may embody several
benefits for workers and even
save lives as many employers are
starting to use computers to
check whether employees are
wearing appropriate safety gears,
goggles and gloves, it may also
endanger live as it increases
pressure to meet deadlines,
statistics and KPI.
technology is not neutral
Lack of transparency
Tracing, scoring, incentivization,
rankings and all the metrics can
be manipulated and repurposed
to infer unspecified
characteristics or to predict
unknown behaviors + systems
that, once designed and
calibrated, run automatically and
gather enormous amounts of
granular data about workers’.
11

12. all AI on us: the surveillance potential of technologies
What is old
AI unique because of
its blend of existing
authoritative
practices, namely
data/people analytics
and algorithmic
governance, in an
optimized manner.
What is new
this “marriage of
convenience”
explains in part why
there are several
concerns about some
of the concrete
functions of this
innovation.
Why do we care
automated choices
for recruiting,
remuneration & even
dismissals, giving
free rein to
discriminatory
biases, exacerbating
inequality.
12

13. a “genetic variation” of managerial prerogative
▸ invasive devices, from badges to tablets, from wearables to
exoskeletons, from cloud software to virtual personal assistant,
constitute a burgeoning terrain for new forms of surveillance
▸ from a legal perspective, they constantly collect, produce, share and
combine data, thus leading to a “genetic variation” of organizational,
control and disciplinary prerogative, considered as the core of the
employment contract, and to informational asymmetry
the prevailing approach is uncritical, and a considerable number of
“users” seem ready to accept a renunciation of privacy so as not to
forego the access to services presented as sources of connection,
optimization, convenience, and pleasure
13

14. A question worth asking is whether authority
today is the same as authority in the past.
The scope of application of provisions on
data protection – based on an “analogue”
understanding of technology – in several civil
law jurisdictions may fall short in providing
an up-to-date model, capable of coping with
unforeseeable technological advances.
14
#1 Research questions

15. Assessing the effects of AI and algorithms
on employment relationships, with a view to
understanding the way how legal and social
institutions should act, react or adapt to a
potential experience of unprecedented “total
surveillance” in the workplace, entrenching
command-and-control relationships between
management and workers.
15
#2 Research questions

16. Examining whether and to what extent the
current legal framework – in EU
countries/civil law systems – is suited to
accommodate and regulate the “augmented”
magnitude of control power: is there a need
for new legislation or is a more effective
enforcement of existing regulation enough?
How can competing interests be reconciled?
16
#3 Research questions

17. 2. Europe taking the lead
“GDPR” and the balance between
companies’ legitimate interests &
privacy expectations of employees
17

18. an “integrated” overview and the “Brussels effect”
▸ the “integrated” study of the General Data Protection Regulation (GDPR)
→ a step forward, leading to a homogenization of national models
instead of promoting a mere harmonization
▹ hailed as one of the best examples of the so-called “Brussels
effect”, the “global power that the EU is exercising through its legal
institutions and standards, successfully export[ing] that influence to
the rest of the world” (Europeanization).
▹ international companies decided to voluntarily implement such
provisions globally → international convergence
▹ the law extends its reach beyond the boundaries of the EU to any
company processing the data of EU citizens
18

19. what’s new
▸ adopted in April 2016 and entered into force in May 2018
▸ Regulation (EU) 2016/679 does not pursue anymore primarily
commercial or economic interests
▸ Recital 4 of GDPR states that the right to the protection of personal data
“is not an absolute right” and “must be considered in relation to its
function in society and be balanced against other fundamental rights, in
accordance with the principle of proportionality”
▸ personal data “which [has] undergone pseudonymisation, which could be
attributed to a natural person by the use of additional information”
should be considered as falling within its scope
19

20. delegated powers
▸ Art. 88 of the GDPR specifies that Member States “may, by law or by
collective agreements, provide for more specific rules to ensure the
protection of the rights and freedoms in respect of the processing of
employees’ personal data in the employment context […]”.
▸ Room for maneuver for the design of “integrative legislation designed to
respond to the risk connected to big data analytics in the employment
relationship”, by devising procedural rules more incisively.
▸ Whether or not social partners would seize this opportunity by imposing
their agenda to the rule-makers still has to be determined
▹ collective bargaining may well represent an effective and flexible way to
“negotiate” the digital transformation of work
20

21. definitions
▸ “processing” = “any operation or set of operations which is performed on
personal data or on sets of personal data, whether or not by automated
means, such as collection, recording, organisation, structuring, storage,
adaptation or alteration, retrieval, consultation, use, disclosure by
transmission, dissemination or otherwise making available, alignment or
combination, restriction, erasure or destruction” (Art. 4, 1 and 2).
▸ the concept of personal data is broad, defined as “any information
relating to an identified or identifiable natural person (‘data subject’)”.
▸ only personal data containing “anonymous information, […] which does
not relate to an identified or identifiable natural person” are excluded
from the scope of the GDPR
21

22. complementary texts
GDPR
focus on a limited
number of strictly
labor-related
provisions
Opinion 2/2017
on data
protection at
work, adopted
in June 2017*
Guidelines on
automated
decision
making &
profiling for the
purposes of
GDPR
22
* 9 scenarios: recruitment process, in-employment screening, monitoring ICT usage both at and outside the
workplace (e.g. home and remote working, “bring your own device” practices, wearable devices), monitoring of
time and attendance, or through video systems, vehicle applications and processing involving the transfer of
employee data to third parties or international operators
Independent EU Advisory Body on Data Protection and Privacy (Article 29 Working Party, “WP29”), comprised of
the heads of the national data protection authorities, that anticipate the future work of its successor, the European
Data Protection Board, in issuing guidelines, recommendations and best practices

23. main principles
▸ Art. 5(1)(a) processing must be:
▹ lawful,
▹ fair,
▹ transparent
▸ Other principles incorporated:
▹ purpose limitation,
▹ data minimization,
▹ accuracy,
▹ storage limitation,
▹ integrity,
▹ confidentiality,
▹ accountability.
23
▸ Art. 6, conditions of lawfulness:
▹ consent by the data subject,
▹ necessity in a contract,
▹ compliance with a legal obligation,
▹ protection of vital interest of the
data subject,
▹ performance of a task carried out
in the public interest or in the
exercise of official authority
vested in the controller,
▹ legitimate interest, provided that
the interests or fundamental rights
and freedom of the data subject
are not jeopardized

24. consent in the context of an employment relationship
▸ as regards data protection in the context of an employment relationship,
according to Article 7, “consent” is not in itself sufficient
▸ the requirement is reinforced by obligations of intelligibility, clarity and
transparency with respect to the modalities of the request and by
strengthening the right to withdraw consent (art. 7(2)(3)).
▸ in particular, in order to assess the genuineness of such a consent,
“utmost account shall be taken of whether, inter alia, the performance of
a contract is conditional on consent”.
▹ as stated in the WP29 Opinion 2/2017’s introduction, “employees
are seldom in a position to freely give, refuse or revoke consent”
24

25. consent in the context of an employment relationship
▸ both consent and a legitimate interest are in itself not sufficient to
override the rights and freedoms of employees
▸ a proportionality test should be undertaken prior to its commencement
to consider whether the processing is necessary to achieve a legitimate
purpose, whether the processing outweighs the data protection right, as
well as the measures that have to be taken to ensure that infringements
of the rights to private life and secrecy of communications are limited to
a minimum (“in the least intrusive manner possible and targeted to the
specific area of risk”)
▸ this can form part of a Data Protection Impact Assessment (DPIA),
according to art. 35 of the GDPR
25

26. key pitfalls and the risk of circumventing the GDPR
▸ many companies have reacted to restrictions based on reinforced
consent and ban on profiling:
▹ by asking individuals to consent to various uses of their personal
data for very widely defined purposes, by getting data subjects to
accept a broadly phrased consent agreement in the first instance –
a practice which has become more questionable
▹ justifying the use of data for statistical purposes, which constitute
an explicitly permitted reuse of data without explicitly abandoning
the purpose limitation principle (“for archiving purposes in the public
interest, scientific or historical research purposes or statistical
purposes”, art. 5(1)(b)) + implementing “appropriate safeguards”
26

27. 2.1 automated processes
individual decision-making,
the GDPR and AI:
premises and promises
of a complicated relationship
27

28. inferential analytics
▸ inferential analytics, one of the strongest AI applications aimed at
deducing conducts by simply extrapolating patterns and recurrences
from large amounts of data, seem largely unregulated
▹ an unintelligible “black box”, which is indented to keep most workers
in the darkness as regards tech-driven strategies
▹ although partially autonomous, they answer to specific
organizational needs and reflect managerial preferences
▹ they are increasingly obscure and unaccountable, yet consequential
the answer? Article 22 regulating “automated individual
decision-making” processes
28

29. article 22 (automatization of organizational procedures)
▸ this is probably the most forward-looking chapter of the Regulation
▸ the Regulation ought to be understood as precluding “a decision based
solely on automated processing, […], which produces legal effects
concerning [the data subject] or similarly significantly affects him or her”
▸ the data subject, i.e. the worker, has the right not to be subject to
decisions “based solely on automated processing, including profiling,
which produces legal effects”
▹ Article 4(4) defines “profiling” as “any form of automated processing of personal data
consisting of the use of personal data to evaluate certain personal aspects relating to a
natural person, in particular to analyse or predict aspects concerning that natural person’s
performance at work, economic situation, health, personal preferences, interests,
reliability, behaviour, location or movements”.
29

30. automated decision-making processes at work
▸ Article 22 does not apply in the case when the automated process
▹ “(a) is necessary for entering into, or performance of, a contract
between the data subject and a data controller”,
▹ (b) when authorized by Union or Member State,
▹ “(c) is based on the data subject’s explicit consent”.
▸ in the (a) and (c) case, the data controller “shall implement suitable
measures to safeguard the data subject’s rights, freedoms & interests”.
▸ if the decision is based on “special” categories of data as defined in art.
9 of the GDPR (i.e. sensitive data), automated decision-making
processes are only allowed on the basis of explicit consent or substantial
public interest, as long as safeguards are put in place
30

31. “the right to obtain human intervention”
▸ under article 22(3), “the right to obtain human intervention on the part of
the controller, to express his or her point of view and to contest the
decision” is established
▹ however, in a machine learning or AI context, “it is not clear who this ‘human’
should be and how to review a process that may have been based on third
party algorithms, pre-learned models or data sets (individuals’ personal data)
or on opaque machine learning models”
▹ explanation may not be feasible in situations where decisions are taken in
response to data in real time
▸ the question arises of how individuals, “who have differing levels of
comprehension”, could access, understand and challenge the info
31

32. 3. Control power
Digital surveillance and
employee monitoring
in France, Germany and Italy
32

33. rationalizing the bureaucratic power of employers
▸ Employment law has been conceived as a set of rules aimed at
rationalizing the managerial prerogative, including surveillance authority
(watchfulness of management), since its emergence
▸ the issue of “humanizing” the bureaucratic power of employers by
means of mandatory provisions or collectively taken countermeasures is
a defining feature of labor law in various legal systems all over Europe,
the very core of labour regulation
→ the traditional legal arsenal regulating the monitoring power of
employer and the right to privacy of employees, compounded by the
most recent regulatory interventions, such as the EU General Data
Protection Regulation, is a starting point
33

34. a comparative approach: DE, FR, IT
34

35. ▸ general right of personality (allgemeines Persönlichkeitsrecht)
▹ important limit to the question of which measures are permitted and
which are prohibited when monitoring an employee
▹ not expressly mentioned in the German Constitution (Grundgesetz,
GG) but represents the consolidated interpretative result of the
principle of human dignity (it shall be “inviolable” – unverletztlich)
and the right to the free development of personality
▸ Federal Constitutional Court emphasises the openness of the general
right of personality to development, which is why a conclusive definition
of the general right of personality has deliberately not yet been made
35
Germany

36. ▸ monitoring of employees at the workplace is only lawful under strict
limits → Data Protection Act (BDSG):
1) Interferences with the right of personality must be justified by
legitimate interests of the employer, other holders of fundamental
rights or other important objectives;
2) expressly permits the processing of employees’ data on the basis of
“collective agreements” (Kollektivvereinbarungen and
Betriebsvereinbarungen) → cannot justify a violation of the
fundamental general right of personality.
→ Works Councils must be involved on automated processes of personal data +
data protection officer appointed if more than 9 employees involved
36
Germany

37. Germany
▸ BUT great relevance of the fundamental right perspective:
Prohibition of total surveillance
Secret surveillance only exceptionally permitted
No excessive pressure to adapt
37

38. ▸ Several provisions of the French Labor Code (“Code du travail”) and of
the Law “Technologies and Freedoms” (“Informatique et libertés”, passed
following the Lyon Caen’s report) define a framework for conditions and
restrictions on the use of technologies at the workplace
▹ French workers have a fundamental right to private lige life (“le droit à une vie
personnelle”) → the right to privacy
▹ any restriction of the general right to privacy may only be justified if it is
aimed at the assessment of the work done by employees
▹ no system of surveillance or data collection may be installed without prior
notice to the employees and to the employees’ representatives (Vigneau)
▹ evidence collected without information are illicit and cannot be used for the
purposes of a disciplinary case
38
France

39. France
▸ Three principles in the French Civil Code:
▹ transparency or loyalty;
▹ proportionality;
▹ relevance.
▸ If the employer wishes to install a surveillance device (e.g. video,
geolocalization et similia), a mandatory conciliation procedure must be
followed (double information, individual and collective, of the workforce.
▹ the employee representatives (works council and health and safety
committee) must be consulted, the employees must be informed
▹ tools performing or allowing personal data processing are to be “declared” to
the National Commission of Data Processing and Freedoms (“Commission
nationale de l’informatique et des libertés”, CNIL)
39

40. Italy
▸ Article 41 Italian Constitution: private economic initiative is free. It may
not be carried out in conflict with social utility or in such a way as to
cause damage to security, freedom or human dignity
▹ Principle of human dignity often used by case law to limit the employer’s
private power
▸ Article 4 of the Workers’ Statute → first formulation in 1970
▹ anticipated the issues of protection of the employee from the employer
control before any rule on privacy existed in Italy
▹ traditionally separated perspectives: the employer’s control on the workplace
/ general privacy issues
▹ recent reform in 2015 deeply changed the disposition, that shall now be
integrated by the Privacy Code
40

41. Italy
1) installation of remote control instruments → audiovisual equipment and
other instruments from which derives even the possibility of remote
control of the activities of workers can be used exclusively for
organizational and production needs, for occupational safety and for the
protection of the company’s assets, and may be installed under the
condition that a collective agreement has been signed or with prior
administrative authorization.
2) The conditions do not apply to tools used by the worker to perform the
working activity and to record the access and presence in the premises of
the company.
41

42. Italy
3) Full possibility to use the information found through instruments
legitimately installed for all purposes related to the employment
relationship provided that the worker is given adequate information on
the methods of use of the instruments and the implementation of
controls and in compliance with the provisions of the so-called Privacy
Code.
▹ Great space for the employer to monitor the employees through
“working instruments”
▹ Anachronistic distinction between “control instruments” and
“working instruments”
42

43. comparative insights on DE, FR, IT
key principle:
awareness of
control
employees shall be
made aware of being
subject of the
(remote) control by
the employer
control power as a
precondition of
disciplinary power
neglected the pure
invasion of the
individual sphere,
reverse the
relationship
is consent
(individual or
collective) enough?
important to define
new rules on consent
that may be insincere
in the context of
employment
43

44. 4. Final remarks
From augmented power
to enhanced freedom
44

45. is GDPR already obsolete?
▸ GDPR seems conceived on an old-fashioned understanding of how data
are used, in turn based on a three-phase system (as classified by Oosten,
acquisition, analysis, and application) and traditional categorisations
(personal, sensitive, ...)
▸ Wacther has leveled well-founded criticism at the GDPR, “focus[ing] too
much on the input stage, meaning when data is collected, but not enough
on how it is assessed”
given the pace of change, employers may find themselves being
able to make connections they had not anticipated or disclosed:
once the data is lawfully obtained, very little control or is reserved to
inferential analytics, which remains a “no man’s land”
45

46. “despite he myth of expanded independence,
AI impacts on freedom, privacy, but also
autonomy and moral reasoning, which is
much more relevant in a society in which
the traditionally separation between private
and work life is dissolving.

47. the way forward
▸ although digital transformation advocates postulate the obsolescence of
existing legal categories, social institutions are places in which forces
are in equilibrium between “freedom and equality, risk and solidarity,
efficiency and sociality” (Romagnoli, 2003)
▸ since social dynamics and implications are neither neutral nor
exogenous, there is a need to create an atmosphere of trust and
accountability regarding the development and use of AI and algorithms
if some of the current results are dystopian, guiding the process of
digital transformation in an informed, fair and sustainable way is the
only recipe for meeting the most ambitious and emancipating
expectations of both the workplace and contemporary society.
47

48. 48
THANKS!
Any questions?
antonio.aloisi@eui.eu
gramano@jur.uni-frankfurt.de