GDB UPC is capable of debugging UPC programs compiled with GCC UPC compiler.
GCC UPC Compiler Options
The following compile time options must be used to get the full benefit of debugging the object code with GDB UPC:
-g -O0 -dwarf-2-upc
"-dwarf-2-upc" option turns on DWARF2 debugging records with UPC extensions. Static number of UPC threads (-fupc-threads-N), as well as other compiler options, can also be specified during compile time.
GDB UPC Debugger OptionsThe following debugging option must be specified for proper usage of GDB UPC.
set pagination off
set target-async on
set detach-on-fork off
set non-stop on
set upcstartgate on
set target-async on
The use of asynchronous commands is enabled. GDB UPC goes a step further and does not require the user to enter "&" sign at the end of the async command.
set detach-on-fork off
This option controls the ability of GDB UPC to control both parent and child processes. GDB UPC requires the ability to control all the created processes in the system. This option is important if UPC threads are mapped directly into Unix processes.
set non-stop on
The option controls the behaviors of debugged threads. It allows for debugging a thread (or threads) while other threads are running. Even though GDB UPC can work in all-stop mode, this feature is required for GDB UPC features related to collective breakpoints and single stepping.
set upcstartgate on
The option that allows for UPC threads synchronization at start-up. It is ON by default and meaningful only when UPC threads are mapped into multiple UNIC processes.
set pagination off
Option to turn off GDB output pagination.
Environment variable UDA_GUPC_PLUGIN_LIBRARY must be set to an exact location of UDA plugin shared library. It is loaded by GDB UPC and used to interpret shared address space mappings.
csh:
setenv UDA_GUPC_PLUGIN_LIBRARY [...]/uda-plugin.so
UDA plugin is part of the UDA server package.
Sample Debugging ScriptThe following debugging script sets up appropriate debugging environment for GDB UPC. It saves all the GDB UPC required options in ".gdb-upc-init" file that is specified as the start-up for GDB UPC:
#!/bin/sh
[ -n "$GDB_UPC" ] \
|| export GDB_UPC=/usr/local/gdb-upc/bin/gdb
[ -n "$UDA_GUPC_PLUGIN_LIBRARY" ] \
|| export UDA_GUPC_PLUGIN_LIBRARY=/usr/local/gdb-upc/uda-plugin/uda-plugin.so
if [ '!' -f "$GDB_UPC" ]; then
echo "not found: $GDB_UPC" >&2
exit 2
fi
if [ '!' -x "$GDB_UPC" ]; then
echo "not executable: $GDB_UPC" >&2
exit 2
fi
if !"$GDB_UPC" -v 2>&1 | grep -sq 'GDB UPC'; then
echo "not GDB UPC: $GDB_UPC" >&2
exit 2
fi
if [ ! -f "$UDA_GUPC_PLUGIN_LIBRARY" ]; then
echo "UDA plugin not found: ${UDA_GUPC_PLUGIN_LIBRARY}" >&2
exit 2
fi
if ! file "$UDA_GUPC_PLUGIN_LIBRARY" | grep -sqi 'shared object'; then
echo "UDA plugin not a shared object: ${UDA_GUPC_PLUGIN_LIBRARY}" >&2
exit 2
fi
if [ ! -w . ]; then
echo "can't write into current working directory: $PWD" >&2
exit 2
fi
if ! cat > ./.gdb-upc-init << EOF
set pagination off
set target-async on
set detach-on-fork off
set non-stop on
set upcstartgate on
EOF
then
echo "can't write to: $PWD/.gdb-upc-init" >&2
exit 2
fi
${GDB_UPC} -x ./.gdb-upc-init "$@"
rm -f ./.gdb-upc-init
The above script can be downloaded from the GDB UPC download area.
UPC Debugging Start-UpUPC program start-up is an important aspect of GDB-UPC debugging when multiple Unix processes are used for UPC thread mappings. Start-up mechanism implemented by GCC UPC run-time is similar to the one implemented by Totalview debugger for MPI program start-up. A debugging gate, asserted by debugger, is used to make sure that all the threads are started correctly and all of them reached the gate. At that time debugger lowers the gate and passes control to the user.
For UPC programs with only one thread or for pthreads implementation of the run-time, debugging gate is not used. More information can be found in sample debugging session’s page.
Options and commands used in program start-up:
set upcstartgate [on|off]
This sets the debugging gate option. If ON, GDB UPC asserts the debugging gate and threads are left running on the gate until released by the user command upc-sync. By default this option is ON.
upc-sync
Command to stop all threads, lift the debugging gate, and switch debugging into UPC mode where only UPC threads are visible, commands are always asynchronous, and collective breakpoints and single stepping is active. Use only at start-up, after the "run" command and after all the threads started.
The following shows four UPC threads starting and being synchronized on the debugging gate (assuming that debugging script described above is used):
# debug-script upc_example
[...]
(gdb) r
Starting program: /eng/upc/dev/nenad/gdb-upc-dev/examples/upc-example
upc_lang: using GUPC plugin.
UPC Threads sync debugging is on.
Use upc-sync command to stop all threads and lift the debugging gate.
[New UPC Thread 0]
[New UPC Thread 1]
[New UPC Thread 2]
[New UPC Thread 3]
(gdb) upc-sync
UPC Mode activated.
(gdb-upc)
At this point GDB UPC's UPC mode is turned on.
NOTE: As GDB UPC works in asynchronous mode, command prompt might be shown out of order (e.g. before any thread's announce message). Make sure you don't simple press enter to get the prompt as GDB UPC will execute your previous command.
GDB UPC - UPC Mode (upcmode)GDB UPC enters a UPC mode (upcmode) once"upc-sync" command is executed. In case of multiple UNIX processes this is done once all the processes were synchronized, while in the single thread or pthreads environments "upc-sync" command should be executed on the first breakpoint.
The following distinguish upcmode from the regular GDB mode:
- commands are asynchronous without a need from user to enter '&' at the end of the command
- The info commands show only UPC threads. GCC UPC run-time contains an extra thread (monitor thread) that is controlled by GDB UPC and visible in gdb mode of operation.
- The execution commands like continue, next, step are collective commands and apply to all threads being debugged. There is no need for user to specify "-a" option for continue.
- The commands that use or show thread numbers use UPC thread numbers in upcmode. For example UPC thread 0 is referenced as 0 thread in upcmode and tread number 2 in gdb mode.
- The "thread apply all" command is applied to all UPC threads. A few aliases to this command are added.
- By default "upc-sync" command turns on collective breakpoint and stepping features
Available commands:
set upcmode [on|off]
Turn upcmode ON or OFF. This is useful in cases where a particular thread requires debugging while the others are stopped. For example, all the threads run to a certain point, upcmode is turned OFF, and particular thread is being selected for debugging.
show upcmode
Show the state of upcmode.
upcall, uall, ua
Various aliases to the "thread apply all" command.
Collective BreakpointsUPC collective breakpoint feature allows the UPC user to insert a breakpoint that is announced to users once all the threads reach it. Placing a breakpoint at the upc_barrier is a good example for collective breakpoint. A warning is issued if multiple threads reach different collective breakpoints which are an indication of an error in the code logic or collective breakpoint was inserted in a thread specific code segment.
Collective breakpoint is only active in upcmode. There is no special collective breakpoint command; instead, regular breakpoint inserted in all threads is treated as a collective one. A breakpoint inserted in particular thread (e.g. b func_call thread 3) is not a collective one.
Execution commands (continue, next, step) have no effects on threads stopped at the collective breakpoint until all threads reach it.
Threads stopped on a collective breakpoint are labeled as such, unless all the threads reached the breakpoint. For example, in the following example a collective breakpoint was inserted at line 27 and at the same time a thread 0 specific breakpoint was inserted at line 25:
[New UPC Thread 0]
[New UPC Thread 1]
[New UPC Thread 2]
[New UPC Thread 3]
(gdb) upc-sync
UPC Mode activated.
(gdb-upc) b upc_main
Breakpoint 1 at 0x4016cd: file upc-example.upc, line 23. (5 locations)
(gdb-upc) c
Continuing.
(gdb-upc) Current language:· auto
The current source language is "auto; currently upc".
Breakpoint 1, main () at upc-example.upc:23
(gdb-upc) b 27
Breakpoint 2 at 0x4016e1: file upc-example.upc, line 27. (4 locations)
(gdb-upc) b 25 thread 0
Breakpoint 3 at 0x4016d7: file upc-example.upc, line 25. (4 locations)
(gdb-upc) c
Continuing.
(gdb-upc)
Breakpoint 3, main () at upc-example.upc:25
25·· ······ printf ("Start.\n");
info thread
· 3 UPC Thread 3 (cb 2)· main () at upc-example.upc:27
· 2 UPC Thread 2 (cb 2)· main () at upc-example.upc:27
· 1 UPC Thread 1 (cb 2)· main () at upc-example.upc:27
* 0 UPC Thread 0· main () at upc-example.upc:25
(gdb-upc)
As example shows, three threads will reach the collective breakpoint, while thread 0 reaches its own breakpoint. Thread info command includes a filed (cb 2) for each thread waiting on the collective breakpoint 2. After simple continue command, thread 0 reaches collective breakpoint and breakpoint is announced to the user.
(gdb-upc) c
Continuing.
Start.
Breakpoint 2, main () at upc-example.upc:27
27·· ·· upc_barrier;
(gdb-upc) info threads
· 3 UPC Thread 3· main () at upc-example.upc:27
· 2 UPC Thread 2· main () at upc-example.upc:27
· 1 UPC Thread 1· main () at upc-example.upc:27
* 0 UPC Thread 0· main () at upc-example.upc:27
(gdb-upc)
Available commands:
set breakpoint collective [on|off]
This turns ON of OFF collective breakpoint feature. There might be a case where the breakpoint announcement from each thread is useful to determine the timing of the program.
show breakpoint collective
This shows the state collective breakpoint feature.
maint collective clear [all|N]
The maintenance command to clear a thread from the collective breakpoint wait. For example, due to the debugging session error a collective breakpoint is inserted in the code segment that is executed by only one thread.
The following example illustrates the usage of "maint collective clear" command:
23······· if (!MYTHREAD)
24········· {
25··········· printf ("Start.\n");
26········· }
27······· upc_barrier;
(gdb-upc) b 27
Breakpoint 2 at 0x4016e1: file upc-example.upc, line 27. (4 locations)
(gdb-upc) b 25
Breakpoint 3 at 0x4016d7: file upc-example.upc, line 25. (4 locations)
(gdb-upc) c
Continuing.
WARNING: Collective breakpoint missmatch: thread 0
Breakpoint 3, main () at upc-example.upc:25
25··········· printf ("Start.\n");
WARNING: Collective breakpoint missmatch: thread 3
Breakpoint 2, main () at upc-example.upc:27
27······· upc_barrier;
(gdb upc) info thread·····
3 UPC Thread 3 (cb 2)· main () at upc-example.upc:27·
2 UPC Thread 2 (cb 2)· main () at upc-example.upc:27·
1 UPC Thread 1 (cb 2)· main () at upc-example.upc:27
* 0 UPC Thread 0 (cb 3)· main () at upc-example.upc:25
(gdb-upc) maint collective clear 3
(gdb-upc) info thread·
3 UPC Thread 3 (cb 2)· main () at upc-example.upc:27
· 2 UPC Thread 2 (cb 2)· main () at upc-example.upc:27
· 1 UPC Thread 1 (cb 2)· main () at upc-example.upc:27
* 0 UPC Thread 0· main () at upc-example.upc:25
(gdb-upc) c
Continuing.
(gdb-upc) Start.
Breakpoint 2, main () at upc-example.upc:27
27······· upc_barrier;
In the above thread 0 reaches breakpoint 3 while the others reach breakpoint 2. At this point continuing debugging is not possible as execution commands have no effect on threads waiting on collective breakpoints. Maintenance command is used to release thread 0 from breakpoint 3 after which thread 0 was able to continue and reach collective breakpoint 2.
Collective Stepping
Collective stepping feature is similar to the breakpoints above, all the threads are stepping together over the code and only one thread is reporting the user. It is particularly useful if the code stepping is the same on all threads. Once code diverges, user must become careful as reporting thread (in most of the cases thread 0) does not execute the same code as everybody else (e.g. thread apply all next might be more useful command as it will step all the threads and show their results).
Available commands:
set breakpoint collective_stepping [on|off]
Turns ON or OFF collective stepping independently of upcmode.
show breakpoint collective_stepping
Shows the state of collective stepping feature.
