Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib/syscall-shim: Support for clone implementation #565

Closed
wants to merge 10 commits into from

Conversation

skuenzer
Copy link
Member

@skuenzer skuenzer commented Oct 4, 2022

This PR introduces support for implementing system calls that 1) require to know the system call return address, like clone, 2) support caller TLSes that are different to Unikraft-internal managed TLS, 3) improve the debug printing options, and 4) implement missed features to map system calls statically with musl.

GitHub-Depends-On: #564

Copy link
Member

@dragosargint dragosargint left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have only one suggestion, to change the uk_pr_debug() to avoid warnings. Other than that here are some tests that I used [1]
[1] https://github.com/dragosargint/test

lib/syscall_shim/uk_syscall_binary.c Show resolved Hide resolved
@razvand razvand added this to the v0.12.0 (Epimetheus) milestone Oct 21, 2022
@razvand razvand self-assigned this Oct 21, 2022
@razvand razvand requested a review from adinasm November 2, 2022 08:38
@adinasm adinasm mentioned this pull request Nov 6, 2022
4 tasks
@nderjung nderjung marked this pull request as draft November 8, 2022 11:57
@nderjung nderjung changed the title [DRAFT] lib/syscall-shim: Support for clone implementation lib/syscall-shim: Support for clone implementation Nov 8, 2022
Copy link
Member

@craciunoiuc craciunoiuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this PR actually contains only the last 2 commits.

Please break up the last commit in 2 (and, as such, shorten the commit message).

Other than this, it looks fine.

@skuenzer skuenzer force-pushed the skuenzer/syscallshim-clone branch 2 times, most recently from fca4b70 to 2240d52 Compare November 16, 2022 11:26
Copy link
Member

@craciunoiuc craciunoiuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@skuenzer

Final round of comments from my side, I'll approve after this :)

lib/vfscore/stdio.c Outdated Show resolved Hide resolved
lib/vfscore/stdio.c Outdated Show resolved Hide resolved
lib/syscall_shim/include/uk/syscall.h Show resolved Hide resolved
lib/syscall_shim/include/uk/syscall.h Show resolved Hide resolved
lib/syscall_shim/include/uk/syscall.h Show resolved Hide resolved
Copy link
Member

@dragosargint dragosargint left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I retested this PR, works well! Thanks, @skuenzer!

Reviewed-by: Dragos Iulian Argint [email protected]

Copy link
Contributor

@adinasm adinasm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Reviewed-by: Adina Smeu [email protected]

skuenzer and others added 9 commits November 25, 2022 14:35
Uses the build system's ability for generating source files from AWK
programs. This simplifies the library's `Makefile.uk` and the re-
generation is properly triggered on dependency changes.
Because generated header files are stored in a different destination
path, the same change cannot be performed for these files, at the
moment.

Signed-off-by: Simon Kuenzer <[email protected]>
Compiles the binary system call handler code as ISR-safe. Additionally,
the extended context is saved and restored of the system call caller on
each binary system call. The reason is that extended register sets maybe
used by the system call implementations and the Linux system call
calling convention assumes that caller programs do not have to save the
state themselves.

Signed-off-by: Simon Kuenzer <[email protected]>
Introduces an option to save & restore the TLS pointer on binary system
calls. Unikraft's internal TLS pointer is loaded before the system call
is handled. This option allows the usage of different TLS pointers
within the system caller program. We name such programs, userland
programs.

Signed-off-by: Simon Kuenzer <[email protected]>
Instead of initializing the default set of file descriptors with a
constructor function, we initialize them during early init. At this
point of boot time, the heap, scheduling, and TLS are initialized.
Because `vfscore` uses the `dup` system call, we introduce this commit
in preparation for the fact that a TLS may be required to perform a
system call.

Signed-off-by: Simon Kuenzer <[email protected]>
This commit introduces system call return address tracking, which
is needed by some system call implementations (like `SYS_CLONE`). Such
implementations can query the return address with
`uk_syscall_return_addr()`.

Checkpatch-Ignore: MULTISTATEMENT_MACRO_USE_DO_WHILE
Signed-off-by: Simon Kuenzer <[email protected]>
Variadic system calls are system calls with a variable number of
arguments. Some of them become optional or are only needed depending on
a given flag or a command number.
This commit introduces the support for statically mapping such
system calls at compile time. For this purpose, the
`uk_syscall_(r_)static()` macros are evaluating the number of given
arguments and will automatically auto-complete missing arguments with
zeros or remove additional arguments that aren't handled by the
particular system call implementation.

Signed-off-by: Simon Kuenzer <[email protected]>
If `CONFIG_LIBSYSCALL_SHIM_HANDLER_ULTLS` is configured, system call
handlers can retrieve the caller's TLS pointer with
`uk_syscall_ultlsp()` if the system call was called by a binary system
call.
Please note that only if the system call was called by a binary system
call and the TLS pointer of the caller is different to the Unikraft-TLS
pointer, this function will return the caller's TLS pointer.

Signed-off-by: Simon Kuenzer <[email protected]>
The (internal) symbol declarations of `_uk_vprintd()` and `_uk_printd()`
are always provided. This allows rare cases where libraries have their
own decision logics when and which debug message to compile in.

Signed-off-by: Simon Kuenzer <[email protected]>
This commit introduces two options for printing a debug message 1) when
a system call handler is invoked and 2) when a binary system call is
handled.

Signed-off-by: Simon Kuenzer <[email protected]>
Copy link
Member

@craciunoiuc craciunoiuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good here. Thanks!

Reviewed-by: Cezar Craciunoiu [email protected]

@skuenzer
Copy link
Member Author

All good here. Thanks!

Reviewed-by: Cezar Craciunoiu [email protected]

Thanks a lot to you all for checking it!

@unikraft-bot
Copy link
Member

Checkpatch passed

Beep boop! I ran Unikraft's checkpatch.pl support script on your pull request and it all looks good!

SHA commit checkpatch
4934b69 lib/syscall-shim: Binary system call handler to separate C file
b68eed8 lib/syscall_shim: Use build system's AWK rules for generating sources
a1ed157 lib/syscall-shim: Save & restore ECTX on binary system calls
1ce1c66 lib/syscall-shim: Save & restore TLS pointer on binary system calls
2f50b60 lib/vfscore: Initialize stdout, stdin, stderr as early during init
2d2179f lib/syscall_shim: Caller return address tracking
Checkpatch-Ignore: MULTISTATEMENT_MACRO_USE_DO_WHILE
52b0d5e lib/syscall_shim: Compile-time mapping of variadic system calls
3a5bdcf lib/syscall_shim: TLS pointer of caller
941ac19 lib/ukdebug: Always provide `_uk_vprintd()`, `_uk_printd()`
da103d0 lib/syscall_shim: Option to show debug messages on system calls only

Copy link
Contributor

@razvand razvand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @skuenzer

Reviewed-by: Razvan Deaconescu [email protected]
Approved-by: Razvan Deaconescu [email protected]

unikraft-bot pushed a commit that referenced this pull request Nov 25, 2022
Uses the build system's ability for generating source files from AWK
programs. This simplifies the library's `Makefile.uk` and the re-
generation is properly triggered on dependency changes.
Because generated header files are stored in a different destination
path, the same change cannot be performed for these files, at the
moment.

Signed-off-by: Simon Kuenzer <[email protected]>
Reviewed-by: Dragos Iulian Argint <[email protected]>
Reviewed-by: Adina Smeu <[email protected]>
Reviewed-by: Cezar Craciunoiu <[email protected]>
Reviewed-by: Razvan Deaconescu <[email protected]>
Approved-by: Razvan Deaconescu <[email protected]>
Tested-by: Unikraft CI <[email protected]>
GitHub-Closes: #565
unikraft-bot pushed a commit that referenced this pull request Nov 25, 2022
Compiles the binary system call handler code as ISR-safe. Additionally,
the extended context is saved and restored of the system call caller on
each binary system call. The reason is that extended register sets maybe
used by the system call implementations and the Linux system call
calling convention assumes that caller programs do not have to save the
state themselves.

Signed-off-by: Simon Kuenzer <[email protected]>
Reviewed-by: Dragos Iulian Argint <[email protected]>
Reviewed-by: Adina Smeu <[email protected]>
Reviewed-by: Cezar Craciunoiu <[email protected]>
Reviewed-by: Razvan Deaconescu <[email protected]>
Approved-by: Razvan Deaconescu <[email protected]>
Tested-by: Unikraft CI <[email protected]>
GitHub-Closes: #565
unikraft-bot pushed a commit that referenced this pull request Nov 25, 2022
Introduces an option to save & restore the TLS pointer on binary system
calls. Unikraft's internal TLS pointer is loaded before the system call
is handled. This option allows the usage of different TLS pointers
within the system caller program. We name such programs, userland
programs.

Signed-off-by: Simon Kuenzer <[email protected]>
Reviewed-by: Dragos Iulian Argint <[email protected]>
Reviewed-by: Adina Smeu <[email protected]>
Reviewed-by: Cezar Craciunoiu <[email protected]>
Reviewed-by: Razvan Deaconescu <[email protected]>
Approved-by: Razvan Deaconescu <[email protected]>
Tested-by: Unikraft CI <[email protected]>
GitHub-Closes: #565
unikraft-bot pushed a commit that referenced this pull request Nov 25, 2022
Instead of initializing the default set of file descriptors with a
constructor function, we initialize them during early init. At this
point of boot time, the heap, scheduling, and TLS are initialized.
Because `vfscore` uses the `dup` system call, we introduce this commit
in preparation for the fact that a TLS may be required to perform a
system call.

Signed-off-by: Simon Kuenzer <[email protected]>
Reviewed-by: Dragos Iulian Argint <[email protected]>
Reviewed-by: Adina Smeu <[email protected]>
Reviewed-by: Cezar Craciunoiu <[email protected]>
Reviewed-by: Razvan Deaconescu <[email protected]>
Approved-by: Razvan Deaconescu <[email protected]>
Tested-by: Unikraft CI <[email protected]>
GitHub-Closes: #565
unikraft-bot pushed a commit that referenced this pull request Nov 25, 2022
This commit introduces system call return address tracking, which
is needed by some system call implementations (like `SYS_CLONE`). Such
implementations can query the return address with
`uk_syscall_return_addr()`.

Checkpatch-Ignore: MULTISTATEMENT_MACRO_USE_DO_WHILE
Signed-off-by: Simon Kuenzer <[email protected]>
Reviewed-by: Dragos Iulian Argint <[email protected]>
Reviewed-by: Adina Smeu <[email protected]>
Reviewed-by: Cezar Craciunoiu <[email protected]>
Reviewed-by: Razvan Deaconescu <[email protected]>
Approved-by: Razvan Deaconescu <[email protected]>
Tested-by: Unikraft CI <[email protected]>
GitHub-Closes: #565
unikraft-bot pushed a commit that referenced this pull request Nov 25, 2022
Variadic system calls are system calls with a variable number of
arguments. Some of them become optional or are only needed depending on
a given flag or a command number.
This commit introduces the support for statically mapping such
system calls at compile time. For this purpose, the
`uk_syscall_(r_)static()` macros are evaluating the number of given
arguments and will automatically auto-complete missing arguments with
zeros or remove additional arguments that aren't handled by the
particular system call implementation.

Signed-off-by: Simon Kuenzer <[email protected]>
Reviewed-by: Dragos Iulian Argint <[email protected]>
Reviewed-by: Adina Smeu <[email protected]>
Reviewed-by: Cezar Craciunoiu <[email protected]>
Reviewed-by: Razvan Deaconescu <[email protected]>
Approved-by: Razvan Deaconescu <[email protected]>
Tested-by: Unikraft CI <[email protected]>
GitHub-Closes: #565
unikraft-bot pushed a commit that referenced this pull request Nov 25, 2022
If `CONFIG_LIBSYSCALL_SHIM_HANDLER_ULTLS` is configured, system call
handlers can retrieve the caller's TLS pointer with
`uk_syscall_ultlsp()` if the system call was called by a binary system
call.
Please note that only if the system call was called by a binary system
call and the TLS pointer of the caller is different to the Unikraft-TLS
pointer, this function will return the caller's TLS pointer.

Signed-off-by: Simon Kuenzer <[email protected]>
Reviewed-by: Dragos Iulian Argint <[email protected]>
Reviewed-by: Adina Smeu <[email protected]>
Reviewed-by: Cezar Craciunoiu <[email protected]>
Reviewed-by: Razvan Deaconescu <[email protected]>
Approved-by: Razvan Deaconescu <[email protected]>
Tested-by: Unikraft CI <[email protected]>
GitHub-Closes: #565
unikraft-bot pushed a commit that referenced this pull request Nov 25, 2022
The (internal) symbol declarations of `_uk_vprintd()` and `_uk_printd()`
are always provided. This allows rare cases where libraries have their
own decision logics when and which debug message to compile in.

Signed-off-by: Simon Kuenzer <[email protected]>
Reviewed-by: Dragos Iulian Argint <[email protected]>
Reviewed-by: Adina Smeu <[email protected]>
Reviewed-by: Cezar Craciunoiu <[email protected]>
Reviewed-by: Razvan Deaconescu <[email protected]>
Approved-by: Razvan Deaconescu <[email protected]>
Tested-by: Unikraft CI <[email protected]>
GitHub-Closes: #565
unikraft-bot pushed a commit that referenced this pull request Nov 25, 2022
This commit introduces two options for printing a debug message 1) when
a system call handler is invoked and 2) when a binary system call is
handled.

Signed-off-by: Simon Kuenzer <[email protected]>
Reviewed-by: Dragos Iulian Argint <[email protected]>
Reviewed-by: Adina Smeu <[email protected]>
Reviewed-by: Cezar Craciunoiu <[email protected]>
Reviewed-by: Razvan Deaconescu <[email protected]>
Approved-by: Razvan Deaconescu <[email protected]>
Tested-by: Unikraft CI <[email protected]>
GitHub-Closes: #565
@unikraft-bot unikraft-bot added the ci/merged Merged by CI label Nov 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/lib Internal Unikraft Microlibrary ci/merged Merged by CI lang/c Issues or PRs to do with C/C++ lib/syscall_shim lib/ukdebug lib/vfscore VFS Core Interface release-note Denotes a PR that will be considered when it comes time to generate release notes.
Projects
Status: Done
Status: Done!
Development

Successfully merging this pull request may close these issues.

7 participants