Another area in which the use of AI systems deserves special consideration is the access to
and enjoyment of certain essential private and public services and benefits necessary for
people to fully participate in society or to improve one’s standard of living. In particular,
natural persons applying for or receiving essential public assistance benefits and services
from public authorities namely healthcare services, social security benefits, social services
providing protection in cases such as maternity, illness, industrial accidents, dependency or
old age and loss of employment and social and housing assistance, are typically dependent
on those benefits and services and in a vulnerable position in relation to the responsible
authorities. If AI systems are used for determining whether such benefits and services
should be granted, denied, reduced, revoked or reclaimed by authorities, including whether
beneficiaries are legitimately entitled to such benefits or services, those systems may have
a significant impact on persons’ livelihood and may infringe their fundamental rights, such
as the right to social protection, non-discrimination, human dignity or an effective remedy
and should therefore be classified as high-risk. Nonetheless, this Regulation should not
hamper the development and use of innovative approaches in the public administration,
which would stand to benefit from a wider use of compliant and safe AI systems, provided
that those systems do not entail a high risk to legal and natural persons. In addition, AI
systems used to evaluate the credit score or creditworthiness of natural persons should be
classified as high-risk AI systems, since they determine those persons’ access to financial
resources or essential services such as housing, electricity, and telecommunication
services. AI systems used for this purpose may lead to discrimination of persons or groups
and perpetuate historical patterns of discrimination, for example based on racial or ethnic
origins, gender, disabilities, age, sexual orientation, or create new forms of discriminatory
impacts. However, AI systems provided for by Union law for the purpose of detecting
fraud in the offering of financial services and for prudential purposes to calculate credit
institutions’ and insurances undertakings’ capital requirements should not be considered as
high-risk under this Regulation. Moreover, AI systems intended to be used for risk
assessment and pricing in relation to natural persons for health and life insurance can also
have a significant impact on persons’ livelihood and if not duly designed, developed and
used, can infringe their fundamental rights and can lead to serious consequences for
people’s life and health, including financial exclusion and discrimination. Finally, AI
systems used to evaluate and classify emergency calls by natural persons or to dispatch or
establish priority in the dispatching of emergency first response services, including by
police, firefighters and medical aid, as well as of emergency healthcare patient triage
systems, should also be classified as high-risk since they make decisions in very critical
situations for the life and health of persons and their property.