HEPHAISTOS has been created as a team on January 1st, 2013 and as a project team in 2015.
The goal of the project is to set up a generic methodology for the design and evaluation of an adaptable and interactive assistive ecosystem for the elderly and the vulnerable persons that provides furthermore assistance to the helpers, on-demand medical data and may manage emergency situations. More precisely our goals are to develop devices with the following properties:
Assistance will be provided through a network of communicating devices that may be either specifically designed for this task or be just adaptation/instrumentation of daily life objects.
The targeted population is limited to frail people 1 and the assistive devices will have to support the individual autonomy (at home and outdoor) by providing complementary resources in relation with the existing capacities of the person. Personalization and adaptability are key factor of success and acceptance. Our long term goal will be to provide robotized devices for assistance, including smart objects, that may help disabled, elderly and handicapped people in their personal life.
Assistance is a very large field and a single project-team cannot
address all the related issues. Hence HEPHAISTOS will focus on the following
main societal challenges:
Addressing these societal focuses induces the following scientific
objectives:
The societal challenges and the scientific objectives will be supported by experimentation and simulation using our development platforms or external resources.
In terms of methodologies the project will focus on the use and
mathematical developments of symbolic tools (for modeling, design, interval analysis), on interval
analysis (for design, uncertainties management, evaluation), on game
theory (for control, localization, economy of
assistance) and on
control
theory.
Implementation of the algorithms will be performed within the
framework of general purpose software such as Scilab, Maple, Mathematica and the interval analysis part will be based
on the existing library ALIAS, that is
still being developed mostly for internal use.
Experimental work and the development of our own prototypes are strategic for the project as they allow us to validate our theoretical work and to discover new problems that will feed in the long term the theoretical analysis developed by the team members.
Dissemination is also an essential goal of our activity as its background both on the assistance side and on the theoretical activities as our approaches are not sufficiently known in the medical, engineering and academic communities.
In summary HEPHAISTOS has as major research axes assistance robotics, modeling, game theory, interval analysis, robotics and AI (see section 8.1). The coherence of these axes is that interval analysis is a major tool to manage the uncertainties that are inherent to a robotized device, while assistance robotics provides realistic problems which allow us to develop, test and improve our algorithms. Our overall objectives are presented in this document and in a specific page on assistance.
As seen in the overall objectives managing uncertainties is a key point of our research. In the health domain uncertainties is managed with statistics (which explain partly presence of E. Wajnberg in our team) but statistics just give trends while in some cases we will be more interested in the worst case scenario. Interval analysis is an approach that can be used in that case and we constantly improve the foundations of this method.
We are interested in real-valued system solving
(
Solutions are searched within a finite domain (called a box)
which may be either continuous or mixed (i.e. for which some variables
must belong to a continuous range while other variables may
only have values within a discrete set). An important point is that we
aim at finding all the solutions within the domain whenever the
computer arithmetic will allow it: in other words we are looking for
certified solutions. For example, for 0-dimensional system
solving, we will provide a box that contains one, and only one,
solution together with a numerical approximation of this
solution. This solution
may further be refined at will using multi-precision.
The core of our methods is the use of interval analysis that
allows one to manipulate mathematical expressions whose unknowns have interval
values. A basic component of interval analysis is the interval
evaluation of an expression. Given an analytical expression
In other words the interval evaluation provides a lower bound of the
minimum of
For example if
The interval evaluation of an expression has interesting properties:
A major drawback of the interval evaluation is that
Fortunately there are methods that allow one to reduce the
overestimation and the overestimation amount decreases with the width of
the ranges. The latter remark leads to the use of a branch-and-bound
strategy in which for a given box a variable range will be bisected,
thereby creating two new boxes that are stored in a list and
processed later
on. The algorithm is complete if all boxes in the list
have been processed, or if during the process a box generates an answer
to the problem at hand (e.g. if we want to prove that
A generic interval analysis algorithm involves the following steps on the current box 10, 1:
The scope of the HEPHAISTOS project is to address all these steps in order to find the most efficient procedures. Our efforts focus on mathematical developments (adapting classical theorems to interval analysis, proving interval analysis theorems), the use of symbolic computation and formal proofs (a symbolic pre-processing allows one to automatically adapt the solver to the structure of the problem), software implementation and experimental tests (for validation purposes).
Important note:
We have insisted on interval analysis because this is a major
component or our robotics activity. Our theoretical work in
robotics is an analysis of the robotic environment in order to exhibit
proofs on the behavior of the system that may be qualitative (e.g. the
proof that a cable-driven parallel robot with more than 6
non-deformable cables will have at most 6 cables under tension
simultaneously) or
quantitative. In the quantitative case as we are dealing with
realistic and not toy examples (including our own prototypes that are
developed whenever no equivalent hardware is available or to verify our
assumptions) we have to manage problems that are so complex that
analytical solutions are probably out of reach (e.g. the direct
kinematics of parallel robots) and we have to resort to algorithms and
numerical analysis. We are aware of different approaches in numerical
analysis (e.g. some team members were previously involved in teams
devoted to computational geometry and algebraic geometry) but interval
analysis provides us another approach with high flexibility, the
possibility of managing non algebraic problems (e.g. the kinematics of
cable-driven parallel robots with sagging cables, that involves
inverse hyperbolic functions) and to address various types of issues
(system solving, optimization, proof of existence ...). However
whenever needed we will rely as well on statistics, continuation, algebraic
geometry, geometry while AI is currently being investigated (see section 8.1.1).
HEPHAISTOS, as a follow-up of COPRIN, has a long-standing tradition of robotics studies, especially for closed-loop robots 4, especially cable-driven parallel robots. We address theoretical issues with the purpose of obtaining analytical and theoretical solutions, but in many cases only numerical solutions can be obtained due to the complexity of the problem. This approach has motivated the use of interval analysis for two reasons:
Our field of study in robotics focuses on kinematic
issues such as
workspace and singularity analysis, positioning
accuracy, trajectory
planning, reliability, calibration, modularity
management and,
prominently, appropriate design, i.e. determining the dimensioning of
a robot mechanical architecture that guarantees that the real robot
satisfies a given set of
requirements. The
methods that we develop can be used for other robotic problems, see
for example the management of uncertainties in aircraft
design 8.
Our theoretical work must be validated through experiments that are essential for the sake of credibility and, a contrario, experiments will feed our theoretical work. Hence HEPHAISTOS works with partners on the development of real robots but also develops its own prototypes. In the last years we have developed a large number of prototypes and we have extended our development to devices that are not strictly robots but are part of an overall environment for assistance. We benefit here from the development of new miniature, low energy computers with an interface for analog and logical sensors such as the Arduino or the Phidgets. The web page presents all of our prototypes and experimental work. Note that this familiarity with hardware is also used from time to time to develop devices for others INRIA projects and during the Covid crisis our building was instrumented for accurately monitoring CO and CO2 level well before it becomes the norm.
While the methods developed in the project can be used for a very
broad set of application domains (for example we have an activity in
CO2 emission allowances and biology), it is clear that the
size of the project does not allow us to address all of them. Hence
we have decided to focus our applicative activities on mechanism
theory, where we focus on modeling, optimal design and
analysis
of mechanisms. Along the same line our focus is
robotics and especially service
robotics which includes rescue robotics, rehabilitation
and assistive robots for elderly and handicapped people. Although
these topics were new
for us when initiating the project we have spent two years determining
priorities and
guidelines by conducting about 200 interviews with field experts (end-users,
doctors, family and caregivers, institutes), establishing
strong collaboration with them (e.g. with the CHU of Nice-Cimiez) and
putting together an appropriate experimental setup for testing our
solutions.
It must be reminded that we are able to manage a large variety of problems in totally different domains only because interval analysis, game theory and symbolic tools provides us the methodological tools that allow us to address completely a given problem from the formulation and analysis up to the very final step of providing numerical solutions. Hence although we mainly focus on medical and assistance robotics we address also a large number of applications: agriculture, biology, arts, system design to name a few.
Clearly our activities have a negative impact on the environment (travels, hardware orders, ...). Although Sophia-Antipolis is not the best place regarding national travels we have decreased our national and international travel activities while having reduced our global impact at work in different manners (lighting, work mobility, ...). Still we must emphasize that all aspects of our impact have to be taken into account before coercive measures are taken. For example when we travel to a retirement house to install assistive devices, the footprint impact has to be balanced with our social impact and finding the right compromise is not an easy task and the choice is not the responsibility of the team alone. Furthermore human relationships are essential for initiating new research areas and for the time being virtual collaborations and conferences are not very effective for that purpose.
Our works on assistance clearly may have a social impact and we are deeply aware of our ethical responsibilities as illustrated by the activity of the team in ethical committees, our collaboration with the academic law community and our large dissemination effort toward the general audience.
Regarding environmental responsibility energy has been since the very beginning of our project an important topic in our research. Indeed our assistance/health monitoring devices require additional energy source and elderly people may have some difficulties to deal with battery charging. Consequently since the beginning of the project we have focused on low consumption electronic components and most our devices use mechanical energy converter or solar panel to produce most of the energy they need. However the intended benefits of these devices on health, self-esteem and dignity (all issues that are difficult to measure/compare in pure financial terms or with respect to environmental impacts in all their dimensions) have to be taken into account.
Highlights of this year are mainly:
After spending one year at Inria Start-up Studio to explore the market opportunity and the economic feasibility to create a start-up dealing with human activities recognition using non-intrusive measurements, a former team member (
Starting December,
Hybrid methods using symbolic computation, numerical analysis and AI (neural networks) have been developped to solve robotics models. Given the good results obtained on several examples, an important work has been done to extend their genericity to resolution of systems of parametric equations. (see section 6.1)
Other important points includes:
The ALIAS library whose development started in 1998, is a collection of procedures based on interval analysis for systems solving and optimization.
ALIAS is made of two parts:
ALIAS-C++ : the C++ library (87 000 code lines) which is the core of the algorithms
ALIAS-Maple : the Maple interface for ALIAS-C++ (55 000 code lines). This interface allows one to specify a solving problem within Maple and get the results within the same Maple session. The role of this interface is not only to generate the C++ code automatically, but also to perform an analysis of the problem in order to improve the efficiency of the solver. Furthermore, a distributed implementation of the algorithms is available directly within the interface.
A completely new version of our old modular MARIONET-CRANE cable-driven parallel robot prototype is now installed in the robotic hall, for experiments in the field of walking assistance and health monitoring for frail people.
This installation takes benefit and improves the two previous ones, installed for art performances in exhibition center in Amilly (45) and in Mouans-Sartoux (06) - see below 7.3.
Our work in robotics has led us to investigate the use of IA for
finding the real root(s)
of parametric non linear square system of equations i.e. systems
which have
as many unknowns as equations but with
equation coefficients that are functions of
parameters that are assumed to be bounded. Such a system has usually
several solutions and their number
depends on the parameters values and cannot be determined in
advance. An example of such a system is presented in
section 8.1.1 together with the principle of the method. Just as
an outline the method requires to have determined
the solutions (not necessarily
all of them) for a small set linear aspect: two samples (exact solutions from the networks predictions
for a specific system is very fast. Consequently using this
approach is appropriate when a sufficient number of
system instances have to
be solved, this number depending upon the computation time of
alternate methods. Furthermore the process have numerous steps that
can benefit from a distributed implementation both for creating the
networks but also when solving a given system and we are currently
investigating the use of low power consumption AI processors to speed
up the calculation.
We cannot guarantee to
obtain all solutions (although our extensive
tests have shown that in general we
will miss very few solutions)
but we have developed a self-learning process
that may allow to reduce the number of missed solution(s)
as soon as new system instances are
solved . The method is
generic and we are currently developing a software framework that
takes as input
In 2019 during 2 months and in 2022 for 4 months, we have installed two large-scale cable-driven parallel robots as robotics parts of art performances created by A-V. Gasc. These robots were acting like an huge 3D-printer, achieving during the whole exhibitions their assigned task: to continuously deposit several layers of glass micro-beads on a given trajectory at a given velocity. An huge amount of data on the operation of the robots has been collected during these period (about 2 To of data only for the 2022 exhibition) We are currently structuring and curating these data, to make (parts of) them widely available to the robotics community.
As mentioned last year we have started to work on the direct
kinematics (DK) of cable-driven parallel robot (CDPR) having sagging cables
(i.e. being elastic and having their own mass).
The direct kinematics consists in determining the pose(s) of the platform for given cable length
where
We have proposed last year a preliminary version of a DK solver using AI and we have finalized this year a very efficient DK solver 16.
The pose of a CDPR is defined by a vector
level, represents a set of
lengths nodes, on this line
represents the
If two nodes
(edge between them, allowing to establish a directed
graph called the solution graph. In this graph we may identify
the non redundant circuits i.e. a list of successively connected nodes
that contains at most one node at a given level. In robotics terms
these circuits define a path in the aspect.
Poses belonging to the same aspect have the property that any two
poses in the same aspect can be joined by following a
singularity-free path in the
If not enough samples have been obtained for a given
circuit we may add samples just by selecting
a stored sample (
Regarding the neural networks we use multi-layer perceptron (MLP) with
a specific training procedure. It is partly based on a decrease of the
MSE loss but when a new MLP is obtained we run a verification
procedure to obtain the success rate of the MLP. For calculating
this rate the verification
procedure runs the MLP on each sample of the training set and use
the solution prediction of the MLP as guess for a Newton scheme. The
success rate is then calculated as the percentage of samples such that
the verification procedure leads to the expected solution of local solver) will provide the
solution for all the samples of the training sets.
For example for a CDPR having 8 cables (and consequently the DK has 22
equations) we have used 11 initial sets of
We may then assess the quality of this DK solver on verification
sets that provide all DK solutions for a given set
of
After this update all solutions are found for all samples of the verification set. We may then repeat the procedure with a new verification set. However we cannot guarantee that the current DK solver will find all solutions in all case. For example we have found examples where we have assessed 10 verification sets with 100% success rate but for the 11-th one solution has been missed, imposing the creation of a new MLP.
The computation time for this DK solver may be decomposed into the time for establishing the necessary MLPs and the exploitation time for solving one instance of the DK. In a sequential process about 100 hours are necessary to establish the solver while solving one instance of the DK takes about 1 second (to be compared to the 20+ hours that are required when using interval analysis or continuation). However these times may be improved as there are numerous steps for establishing the MLPs that may benefit from a parallel implementation while the local solvers of DK solver may be run in parallel. This year we have investigated the use of the IA processor JETSON Nano but the result was somewhat disappointing because of the limited floating point power of the GPU. Anyhow this AI solver is intended to be used when numerous DK instances have to be solved and here as soon as at least 5 DK instances have to be solved the AI solver is faster than the alternate methods.
Finally as mentioned in section 7.2 the principle of the DK solver is generic and may be used for any parametric square equations system.
The kinematics equations governing the behavior
of CDPR with sagging cables have been presented
presented in section 8.1.1. They are used for control purpose as
they establish the relationship between the actuated variables
(the cable lengths redundant sensing i.e. to complement the cable length
measurements with additional sensors:
These two sensing methods have been implemented on one of our
prototype and have shown to be very effective. Here we investigate
if this sensing redundancy may also allow to assess the wear of the cables
which will decrease the
When controlling the cable lengths to reach a
given pose
First we have to perform a sensitivity analysis i.e to determine
how much change in the
After considering the sensitivity we will have to solve the inverse
problem, i.e. determining the current value of all the
A large number of industrial robots are used mostly only for simple pick-and-place operation, taking an object at a given position and moving it to another one. In most cases this task is performed by serial robot which are not energy efficient (beside the load the robot actuation has to move part of the robot own weight) while using a computer and various electronic boards that are largely under exploited which constitutes a waste of energy and resources. We intend to investigate the use for this task of cable-driven parallel robot, which are 25% more energy efficient, and to design an electro-mechanical mechanism for executing pick-and place trajectories while getting rid of the computer and of most electronic resources.
By lack of manpower, we have reduced our medical activities to human activity recognition. Work on the modular rehabilitation station (described in previous activity reports) has been stopped during one year and will continue in the coming future.
Human activity recognition (HAR) is a major topic in the team. We are focusing on monitoring mobility and displacements (we are not yet interested in recognizing the individual action of our subject) using a sensor-based approach, excluding vision which is intrusive and even prohibited in some places for legal reasons. For that purpose we have in the previous years developed a smart barrier combining redundant passive infrared motion detectors and infrared distance sensors.
Smart barriers have been implemented in Ehpad Valrose, a new retirement house in which a specific infrastructure has been put in place to accommodate research works and in Institut Claude Pompidou, an Alzheimer day care hospital from 2019 to 2020.
These two long term experiments have allowed us to determine that essential points in HAR are to determine what is possible to measure, the sensor types, how to retrieve and process sensor data, how effective are the quality of measurements on a long term basis and the level of monitoring that is acceptable for frail peoples and their helpers while providing significant and reliable data for the medical community in spite of the uncertainties both in the measurements and in the system modeling.
These samples of questions will become central in our work.
We are interested in and currently investigating the use of another type of barriers, based on Lidar's. Measurements produced by lidars are richers and allow a better reliability on the number and the directions of simultaneous crossings.
Three years ago, through an international scientific cooperation with Israeli scientists located at the Ben-Gurion University of the Negev, and as explained in an activity report provided previously (2020), we developed a probabilistic model whose aim was to understand a strange - and up to now not understandable - reproductive behavior of the potter wasp insect Delta dimidiatipenne (Hymenoptera). Females of this insect lay their eggs in mud chambers provisioned with caterpillars they capture to feed their young. Experimental observations indicate that many of the caterpillars collected by this wasp are actually parasitized by a small gregarious parasitoid wasp and are providing a lower amount of food for the wasp progeny. As a result, all players in the interaction perish - the young potter wasp cannot fully exploit the caterpillars and presumably starve to death; and the small parasitoids complete their development, but cannot break out of the mud and remain trapped in the sealed pot. Following such observation, we developed a probabilistic model trying to understand under what environmental conditions such a striking phenomenon (i.e., bringing back to the nest parasitized caterpillars), can be maintained, despite the high cost to all players. After several problems to develop this modeling work (some of them were due to the current political situation in Israel), we were able to submit a manuscript to a good international journal (Behavioural Processes). By the end September, we received a decision (minor revision) and a revised version of the text was resubmitted by mid-November. We are now waiting on the editorial decision from the journal.
Several other activities were developed this (and the previous) years, all lead to publications in international journals: