Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(framework) Add simulation engine SuperExec plugin #3589

Merged
merged 50 commits into from
Jul 15, 2024
Merged

feat(framework) Add simulation engine SuperExec plugin #3589

merged 50 commits into from
Jul 15, 2024

Conversation

jafermarq
Copy link
Contributor

@jafermarq jafermarq commented Jun 12, 2024 •
edited by danieljanes
Loading

Minimal Simulation Engine plugin for the SuperExec.

Built on top of #3555 , expects #3553

Lauch (without TLS):

flower-superexec --executor flwr.superexec.simulation:executor --insecure --executor-config 'num-supernodes=100'

Then start the run from a FAB dir

flwr run --use-superexec

charlesbvll and others added 30 commits June 5, 2024 12:55
@charlesbvll
feat(framework) Add superexec
c0601ba
@charlesbvll
Add debug setup
485616b
@charlesbvll
Improve
12e2d3b
@charlesbvll
Refactoring
156dc4d
@charlesbvll
Rename correctly
53c7106
@charlesbvll
Merge branch 'main' into add-superexec-proto
eb7f6e7
@charlesbvll
Fully working
788cfee
@charlesbvll
Merge branch 'main' into add-superexec-proto
b316114
@charlesbvll
Fix imports ordering
f1809e2
@charlesbvll
Try fixing issues
7cd19cf
@charlesbvll
Fix imports
fd003ba
@charlesbvll
Fix ruff errors
6a4308d
@charlesbvll
Use subscript for Popen
d3e71a9
@charlesbvll
Improve test and servicer
69886ba
@charlesbvll
Fix pylint errors
f818900
@charlesbvll
Try with string type
4708671
@charlesbvll
Define new type
c8b5ef7
@charlesbvll
Use correct syntax
5bee556
@charlesbvll
Remove unused import
6b0f9bc
@charlesbvll
Disable pylint error
f5d7788
@charlesbvll
Remove subscript
d4ad793
@charlesbvll
Ignore type
4e31732
@charlesbvll
Revert driver and server_app changes
a7e990f
@charlesbvll
Correct number of proto files
6f0d3cb
@charlesbvll
Merge branch 'main' into add-superexec-proto
05e63ec
@jafermarq
Merge branch 'main' into add-simulation-engine-plugin
151d0bb
@jafermarq
init
9493097
@jafermarq
Merge branch 'main' into add-simulation-engine-plugin
fba76f0
@jafermarq
Merge branch 'main' into add-simulation-engine-plugin
c4eded8
@jafermarq
merged w/ main; updates
654ec4e
@jafermarq jafermarq marked this pull request as ready for review July 13, 2024 09:32
@jafermarq jafermarq marked this pull request as draft July 13, 2024 15:04
@jafermarq jafermarq marked this pull request as ready for review July 14, 2024 14:45
@danieljanes danieljanes enabled auto-merge (squash) July 14, 2024 20:16
danieljanes
danieljanes reviewed Jul 14, 2024
View reviewed changes
src/py/flwr/superexec/simulation.py Outdated
[
"flower-simulation",
"--client-app",
f"{fab_name}.client:app",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be read from the FAB itself (key: flower.components.clientapp). Users might rename client or app, or move the module in a sub-package.

danieljanes
danieljanes reviewed Jul 14, 2024
View reviewed changes
src/py/flwr/superexec/simulation.py Outdated
"--client-app",
f"{fab_name}.client:app",
"--server-app",
f"{fab_name}.server:app",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be read from the FAB as well.

@danieljanes danieljanes merged commit 5155a62 into main Jul 15, 2024
28 checks passed
@danieljanes danieljanes deleted the add-simulation-engine-plugin branch July 15, 2024 08:12
chongshenng pushed a commit that referenced this pull request Jul 16, 2024
@jafermarq @charlesbvll
@danieljanes @chongshenng
feat(framework) Add simulation engine SuperExec plugin (#3589)
27ee0db 
Co-authored-by: Charles Beauville <charles@flower.ai>
Co-authored-by: Daniel J. Beutel <daniel@flower.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@danieljanes danieljanes danieljanes approved these changes

@tanertopal tanertopal Awaiting requested review from tanertopal tanertopal is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jafermarq @danieljanes @charlesbvll