Skip to content

aidotse/next-word-fedn

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

90 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Federated Learning for Next-Word Prediction

By AI Sweden Young Talents: Västerås.

Navigating this codebase

You can find the training notebook for our final model in ./train

The client code for federated learning is in the ./client

You can find old files and old models in the ./old

We also kept the data in git and is located in ./data

Intro

This project we took in collaboration with Scalout.

We were assigned to use their platform for federated learning and select a relevant industry + usecase to make the project in.

The industry we choosed was Mobile devices and the use case was next-word prediction.

A measurable goal we chose was to make it learn a new word.

Outcome

The final product is a foundation model, web-UI, and a fedn project that works with scaleout.

When presenting this project we managed to train it on hard-coded data from Scaleout's documentation.

After the model had trained in a client and aggregated through Scaleout's platform, we could see a notable difference in how it could predict words related to federated learning.

Before

federated-before

After

federated-after