If I put the @log_sparse decorator above the main function, I do not see the processes from the pool of processes that have been created.

On the contrary, if I put the log_sparse decorator on top of my_function, I do not see what the main process is doing and what happens in between the call of that function and this is what would be really interesting to me!

I launch the profiling from the CLI with:

viztracer --pid_suffix --output_dir ./outputDir --log_sparse test.py\n

What am I missing?

Thanks in advance for your help :)

First of all, if you want to understand your program, you probably should not use log_sparse. That's for cases where you are only interested in the time spent on a specific function. log_sparse by default does not log children functions. So if you put that to main(), it'll only log main(), not any other function calls. That's what sparse means.

As for the UI, it's perfetto so you can check their docs out. I believe all the colors correspond to some CPU usage. The black bar is probably the number function calls starting at that time period. The green one on the process is normally the frequency of function calls as well. Because you only have too few calls, they might not quite make sense. They will be if your trace file is more comprehensive.

Cannot get the main process in vizviewer #407

Answered by gaogaotiantian

MarcoMagl asked this question in Q&A

Cannot get the main process in vizviewer #407

MarcoMagl

Feb 21, 2024 · 2 comments · 1 reply

Answered by gaogaotiantian Return to top

Discussion options

MarcoMagl

Feb 21, 2024

Hi everyone, I would like to use viztracer to better understand why my code does not scale well with multiprocessing. To do so, I started with a simple example. Here is the code: def my_function(a): inside_circle = 0 num_samples = 0 start_time = time.time() # Capture the start time duration = 3 # Duration in second while True: x, y = random.random(), random.random() if x**2 + y**2 <= 1: inside_circle += 1 num_samples += 1 ellapsed = time.time() - start_time if ellapsed > duration: break pid = os.getpid() print(f"Running the function for the {a}th time - Process ID: {pid} took {ellapsed} seconds\n") return 4 * inside_circle / num_samples @log_sparse def main(): with concurrent.futures.ProcessPoolExecutor(max_workers=2) as executor: futures = [executor.submit(my_function, i) for i in range(5)] _ = [f.result() for f in futures] If I put the @log_sparse decorator above the main function, I do not see the processes from the pool of processes that have been created. On the contrary, if I put the log_sparse decorator on top of my_function, I do not see what the main process is doing and what happens in between the call of that function and this is what would be really interesting to me! I launch the profiling from the CLI with: viztracer --pid_suffix --output_dir ./outputDir --log_sparse test.py What am I missing? Thanks in advance for your help :)

1 You must be logged in to vote

Answered by gaogaotiantian Feb 21, 2024

First of all, if you want to understand your program, you probably should not use log_sparse. That's for cases where you are only interested in the time spent on a specific function. log_sparse by default does not log children functions. So if you put that to main(), it'll only log main(), not any other function calls. That's what sparse means.

As for the UI, it's perfetto so you can check their docs out. I believe all the colors correspond to some CPU usage. The black bar is probably the number function calls starting at that time period. The green one on the process is normally the frequency of function calls as well. Because you only have too few calls, they might not quite make sense.…

View full answer

Replies: 2 comments · 1 reply

Comment options

MarcoMagl

Feb 21, 2024

Author

Another couple of questions which come to my mind: Where can I found information on what vizviewer shows? I would like to understand: what can we see on top (black blocks)? what does the green blocks represent? Indeed the function lasts approximately 3 seconds (as I have annotated) so I do not know to what corresponds what we can see below. Thanks in advance!

1 You must be logged in to vote

0 replies

Comment options

gaogaotiantian

Feb 21, 2024

Maintainer

First of all, if you want to understand your program, you probably should not use log_sparse. That's for cases where you are only interested in the time spent on a specific function. log_sparse by default does not log children functions. So if you put that to main(), it'll only log main(), not any other function calls. That's what sparse means. As for the UI, it's perfetto so you can check their docs out. I believe all the colors correspond to some CPU usage. The black bar is probably the number function calls starting at that time period. The green one on the process is normally the frequency of function calls as well. Because you only have too few calls, they might not quite make sense. They will be if your trace file is more comprehensive.

Marked as answer

1 You must be logged in to vote

1 reply

Comment options

MarcoMagl Feb 23, 2024

Author

Thanks for the reply. I used log_sparse in order to limit the size of the output. Even on a toy example without log_sparse, I launched simply with vitracer, I was gettting this kind of output: Total Entries: 1000006 Circular buffer is full, you lost some early data, but you still have the most recent data. If you need more buffer, use "viztracer --tracer_entries <entry_number>" Or, you can try the filter options to filter out some data you don't need use --quiet to shut me up Use the following command to open the report: Now I changed the way I launched to ignore all functions that are frozen and it's better: viztracer --ignore_c_function --ignore_frozen test_calculate_demand.py --parallel-module none Thank you!

Answer selected by MarcoMagl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Category

🙏

Q&A

Labels

None yet

2 participants