patch-2.4.8 linux/Documentation/s390/Debugging390.txt

Next file: linux/Documentation/s390/TAPE
Previous file: linux/Documentation/s390/CommonIO
Back to the patch index
Back to the overall index

diff -u --recursive --new-file v2.4.7/linux/Documentation/s390/Debugging390.txt linux/Documentation/s390/Debugging390.txt
@@ -1,5 +1,5 @@
               
-                          Debugging on Linux for s/390 & zSeries
+                          Debugging on Linux for s/390 & z/Architecture
 			               by
 		Denis Joseph Barrow (djbarrow@de.ibm.com,barrow_dj@yahoo.com)
 		Copyright (C) 2000-2001 IBM Deutschland Entwicklung GmbH, IBM Corporation
@@ -8,7 +8,7 @@
 Overview of Document:
 =====================
 This document is intended to give an good overview of how to debug 
-Linux for s/390 & zSeries it isn't intended as a complete reference & not a
+Linux for s/390 & z/Architecture it isn't intended as a complete reference & not a
 tutorial on the fundamentals of C & assembly, it dosen't go into
 390 IO in any detail. It is intended to compliment the documents in the
 reference section below & any other worthwhile references you get.
@@ -21,46 +21,46 @@
 ========
 Register Set
 Address Spaces on Intel Linux
-Address Spaces on Linux for s/390 & zSeries
-The Linux for s/390 & zSeries Kernel Task Structure
-Register Usage & Stackframes on Linux for s/390 & zSeries
+Address Spaces on Linux for s/390 & z/Architecture
+The Linux for s/390 & z/Architecture Kernel Task Structure
+Register Usage & Stackframes on Linux for s/390 & z/Architecture
 A sample program with comments
-Compiling programs for debugging on Linux for s/390 & zSeries
+Compiling programs for debugging on Linux for s/390 & z/Architecture
 Figuring out gcc compile errors
 Debugging Tools
 objdump
 strace
 Performance Debugging 
 Debugging under VM
-s/390 & zSeries IO Overview
-Debugging IO on s/390 & zSeries under VM
-GDB on s/390 & zSeries
+s/390 & z/Architecture IO Overview
+Debugging IO on s/390 & z/Architecture under VM
+GDB on s/390 & z/Architecture
 Stack chaining in gdb by hand
 Examining core dumps
 ldd
 Debugging modules
 The proc file system
 Starting points for debugging scripting languages etc.
-Tools soon to be available
 SysRq
 References
+Special Thanks
 
 Register Set
 ============
 The current architectures have the following registers.
  
-16  General propose registers, 32 bit on s/390 64 bit on zSeries, r0-r15 or gpr0-gpr15 used for arithmetic & addressing. 
+16  General propose registers, 32 bit on s/390 64 bit on z/Architecture, r0-r15 or gpr0-gpr15 used for arithmetic & addressing. 
 
-16 Control registers, 32 bit on s/390 64 bit on zSeries, ( cr0-cr15 kernel usage only ) used for memory managment,
+16 Control registers, 32 bit on s/390 64 bit on z/Architecture, ( cr0-cr15 kernel usage only ) used for memory managment,
 interrupt control,debugging control etc.
 
-16 Access registers ( ar0-ar15 ) 32 bit on s/390 & zSeries
+16 Access registers ( ar0-ar15 ) 32 bit on s/390 & z/Architecture
 not used by normal programs but potentially could 
 be used as temporary storage. Their main purpose is their 1 to 1
 association with general purpose registers and are used in
 the kernel for copying data between kernel & user address spaces.
-Access register 0 ( & access register 1 on z/Series ( needs 64 bit pointer ) )
-is currently used by the pthread library as a pointer to
+Access register 0 ( & access register 1 on z/Architecture ( needs 64 bit 
+pointer ) ) is currently used by the pthread library as a pointer to
 the current running threads private area.
 
 16 64 bit floating point registers (fp0-fp15 ) IEEE & HFP floating 
@@ -72,7 +72,7 @@
 
 
 The PSW is the most important register on the machine it
-is 64 bit on s/390 & 128 bit on zSeries & serves the roles of 
+is 64 bit on s/390 & 128 bit on z/Architecture & serves the roles of 
 a program counter (pc), condition code register,memory space designator.
 In IBM standard notation I am counting bit 0 as the MSB.
 It has several advantages over a normal program counter
@@ -83,7 +83,7 @@
 currently running at.
 
       Bit           Value
-s/390 zSeries
+s/390 z/Architecture
 0       0     Reserved ( must be 0 ) otherwise specification exception occurs.
 
 1       1     Program Event Recording 1 PER enabled, 
@@ -100,7 +100,7 @@
 
 8-11  8-11    PSW Key used for complex memory protection mechanism not used under linux
 
-12      12    1 on s/390 0 on zSeries
+12      12    1 on s/390 0 on z/Architecture
 
 13      13    Machine Check Mask 1=enable machine check interrupts
 
@@ -175,15 +175,19 @@
                compatible.
  	  
 	  
-Prefix Page
------------	  
+Prefix Page(s)
+--------------	  
 This per cpu memory area is too intimately tied to the processor not to mention.
-It exists between the real addresses 0-4096 on the processor & is exchanged 
-with a page in absolute storage by the set prefix instruction in linux'es startup. 
-This page different on each processor.
-Bytes 0-512 ( 200 hex ) are used by the processor itself for holding such
-information as exception indications & entry points for exceptions.
-Bytes after 0xc00 hex are used by linux for per processor globals.
+It exists between the real addresses 0-4096 on s/390 & 0-8192 z/Architecture & is exchanged 
+with a 1 page on s/390 or 2 pages on z/Architecture in absolute storage by the set 
+prefix instruction in linux'es startup. 
+This page is mapped to a different prefix for each processor in an SMP configuration
+( assuming the os designer is sane of course :-) ).
+Bytes 0-512 ( 200 hex ) on s/390 & 0-512,4096-4544,4604-5119 currently on z/Architecture 
+are used by the processor itself for holding such information as exception indications & 
+entry points for exceptions.
+Bytes after 0xc00 hex are used by linux for per processor globals on s/390 & z/Architecture 
+( there is a gap on z/Architecure too currently between 0xc00 & 1000 which linux uses ).
 The closest thing to this on traditional architectures is the interrupt
 vector table. This is a good thing & does simplify some of the kernel coding
 however it means that we now cannot catch stray NULL pointers in the
@@ -219,9 +223,6 @@
 If using the virtual machine ( VM ) as a debugger it is quite difficult to
 know which user process is running as the address space you are looking at
 could be from any process in the run queue.
-Thankfully you normally get lucky as address spaces don't overlap that &
-you can recognise the code at by cross referencing with a dump made by objdump
-( more about that later ).
 
 The limitation of Intels addressing technique is that the linux
 kernel uses a very simple real address to virtual addressing technique
@@ -240,35 +241,77 @@
 of our 32 bit addresses,however, we use entirely separate address 
 spaces for the user & kernel.
 
-This means we can support 2GB of non Extended RAM, & more
-with the Extended memory managment swap device & 64 Bit
-when it comes along.
+This means we can support 2GB of non Extended RAM on s/390, & more
+with the Extended memory managment swap device & 
+currently 4TB of physical memory currently on z/Architecture.
 
 
-Address Spaces on Linux for S390
-================================
+Address Spaces on Linux for s/390 & z/Architecture
+==================================================
 
 Our addressing scheme is as follows
 
 
-Himem 0x7fffffff 2GB on s/390   *****************          ****************
-2^64 bytes on zSeries           *  User Stack   *          *              *
-				*****************          *              *
-		                *  Shared Libs  *          *              *          
-                                *****************          *              *  
-			        *               *          *    Kernel    *  
-		                *  User Program *          *              *
-		                *   Data BSS    *          *              *
-                                *    Text       *          *              *
-            			*   Sections    *          *              *
-0x00000000                      *****************          ****************
+Himem 0x7fffffff 2GB on s/390    *****************          ****************
+currently 0x3ffffffffff (2^42)-1 *  User Stack   *          *              *
+on z/Architecture.		 *****************          *              *
+		                 *  Shared Libs  *          *              *      
+                                 *****************          *              *  
+			         *               *          *    Kernel    *  
+		                 *  User Program *          *              *
+		                 *   Data BSS    *          *              *
+                                 *    Text       *          *              *
+            			 *   Sections    *          *              *
+0x00000000                       *****************          ****************
 
 This also means that we need to look at the PSW problem state bit
 or the addressing mode to decide whether we are looking at
-user or kernel space. 
+user or kernel space.
+
+Virtual Addresses on s/390 & z/Architecture
+===========================================
+
+A virtual address on s/390 is made up of 3 parts
+The SX ( segment index, roughly corresponding to the PGD & PMD in linux terminology ) 
+being bits 1-11.
+The PX ( page index, corresponding to the page table entry (pte) in linux terminology )
+being bits 12-19. 
+The remaining bits BX (the byte index are the offset in the page )
+i.e. bits 20 to 31.
+
+On z/Architecture in linux we currently make up an address from 4 parts.
+The region index bits (RX) 0-32 we currently use bits 22-32
+The segment index (SX) being bits 33-43
+The page index (PX) being bits  44-51
+The byte index (BX) being bits  52-63
+
+Notes:
+1) s/390 has no PMD so the PMD is really the PGD also.
+A lot of this stuff is defined in pgtable.h.
+
+2) Also seeing as s/390's page indexes are only 1k  in size 
+(bits 12-19 x 4 bytes per pte ) we use 1 ( page 4k )
+to make the best use of memory by updating 4 segment indices 
+entries each time we mess with a PMD & use offsets 
+0,1024,2048 & 3072 in this page as for our segment indexes.
+On z/Architecture our page indexes are now 2k in size
+( bits 12-19 x 8 bytes per pte ) we do a similar trick
+but only mess with 2 segment indices each time we mess with
+a PMD.
+
+3) As z/Architecture supports upto a massive 5-level page table lookup we 
+can only use 3 currently on Linux ( as this is all the generic kernel
+currently supports ) however this may change in future
+this allows us to access ( according to my sums )
+4TB of virtual storage per process i.e.
+4096*512(PTES)*1024(PMDS)*2048(PGD) = 4398046511104 bytes,
+enough for another 2 or 3 of years I think :-).
+to do this we use a region-third-table designation type in
+our address space control registers.
+ 
 
-The Linux for s/390 & zSeries Kernel Task Structure
-===================================================
+The Linux for s/390 & z/Architecture Kernel Task Structure
+==========================================================
 Each process/thread under Linux for S390 has its own kernel task_struct
 defined in linux/include/linux/sched.h
 The S390 on initialisation & resuming of a process on a cpu sets
@@ -287,7 +330,7 @@
             *        ( 4K )        *
 8K aligned  ************************ 
 
-                     zSeries
+                 z/Architecture
             ************************
             *  2 page kernel stack *
 	    *        ( 8K )        *
@@ -298,7 +341,7 @@
 
 What this means is that we don't need to dedicate any register or global variable
 to point to the current running process & can retrieve it with the following
-very simple construct for s/390 & one very similar for zSeries.
+very simple construct for s/390 & one very similar for z/Architecture.
 
 static inline struct task_struct * get_current(void)
 {
@@ -317,8 +360,8 @@
 
 
 
-Register Usage & Stackframes on Linux for s/390 & zSeries
-=========================================================
+Register Usage & Stackframes on Linux for s/390 & z/Architecture
+=================================================================
 Overview:
 ---------
 This is the code that gcc produces at the top & the bottom of
@@ -331,7 +374,7 @@
 limited knowledge of one assembly language.
 
 It should be noted that there are some differences between the
-s/390 & zSeries stack layouts as the zSeries stack layout didn't have
+s/390 & z/Architecture stack layouts as the z/Architecture stack layout didn't have
 to maintain compatibility with older linkage formats.
 
 Glossary:
@@ -367,8 +410,8 @@
 The code generated by the compiler to return to the caller.
 
 frameless-function
-A frameless function in Linux for s390 & zSeries is one which doesn't need 
-more than the register save area ( 96 bytes on s/390, 160 on zSeries )
+A frameless function in Linux for s390 & z/Architecture is one which doesn't 
+need more than the register save area ( 96 bytes on s/390, 160 on z/Architecture )
 given to it by the caller.
 A frameless function never:
 1) Sets up a back chain.
@@ -428,8 +471,8 @@
 }
 
 
-s/390 & zSeries Register usage
-==============================
+s/390 & z/Architecture Register usage
+=====================================
 r0       used by syscalls/assembly                  call-clobbered
 r1	 used by syscalls/assembly                  call-clobbered
 r2       argument 0 / return value 0                call-clobbered
@@ -449,8 +492,8 @@
 
 f0       argument 0 / return value ( float/double ) call-clobbered
 f2       argument 1                                 call-clobbered
-f4       zSeries argument 2                           saved
-f6       zSeries argument 3                           saved
+f4       z/Architecture argument 2                  saved
+f6       z/Architecture argument 3                  saved
 The remaining floating points
 f1,f3,f5 f7-f15 are call-clobbered.
 
@@ -479,12 +522,13 @@
 area if crossing this boundary.
 6) Floating point parameters are mixed with outgoing args
 on the outgoing args area in the order the are passed in as parameters.
-7) Floating point arguments 2 & 3 are saved in the outgoing args area for zSeries
+7) Floating point arguments 2 & 3 are saved in the outgoing args area for 
+z/Architecture
 
 
 Stack Frame Layout
 ------------------
-s/390     zSeries
+s/390     z/Architecture
 0         0             back chain ( a 0 here signifies end of back chain )
 4         8             eos ( end of stack, not used on Linux for S390 used in other linkage formats )
 8         16            glue used in other s/390 linkage formats for saved routine descriptors etc.
@@ -598,8 +642,8 @@
 stack backchain in optimised code as this also causes pipeline stalls, you
 have been warned.
 
-64 bit zSeries code disassembly
--------------------------------
+64 bit z/Architecture code disassembly
+--------------------------------------
 
 If you understand the stuff above you'll understand the stuff
 below too so I'll avoid repeating myself & just say that 
@@ -637,12 +681,14 @@
 
 
 
-Compiling programs for debugging on Linux for s/390 & zSeries
-=============================================================
--gdwarf2 now works & normal -g debugging works much better now
+Compiling programs for debugging on Linux for s/390 & z/Architecture
+====================================================================
+-gdwarf-2 now works it should be considered the default debugging
+format for s/390 & z/Architecture as it is more reliable for debugging
+shared libraries,  normal -g debugging works much better now
 Thanks to the IBM java compiler developers bug reports. 
 
-This is typically done adding/appending the flags -g to the 
+This is typically done adding/appending the flags -g or -gdwarf-2 to the 
 CFLAGS & LDFLAGS variables Makefile of the program concerned.
 
 If using gdb & you would like accurate displays of registers &
@@ -1148,7 +1194,7 @@
 --------------------------------
 D G will display all the gprs
 Adding a extra G to all the commands is neccessary to access the full 64 bit 
-content in VM on zSeries obviously this isn't required for access registers
+content in VM on z/Architecture obviously this isn't required for access registers
 as these are still 32 bit.
 e.g. DGG instead of DG 
 D X will display all the control registers
@@ -1251,7 +1297,7 @@
 tr i pswa <address of main>
 Start the program, if VM drops to CP on what looks like the entry
 point of the main function this is most likely the process you wish to debug.
-Now do a D X13 or D XG13 on zSeries.
+Now do a D X13 or D XG13 on z/Architecture.
 On 31 bit the STD is bits 1-19 ( the STO segment table origin ) 
 & 25-31 ( the STL segment table length ) of CR13.
 now type
@@ -1260,7 +1306,48 @@
 TR I R STD 8F32E1FF 0.7fffffff
 Another very useful variation is
 TR STORE INTO STD <CR13's value> <address range>
+for finding out when a particular variable changes.
 
+An alternative way of finding the STD of a currently running process 
+is to do the following, ( this method is more complex but
+could be quite convient if you aren't updating the kernel much &
+so your kernel structures will stay constant for a reasonable period of
+time ).
+
+grep task /proc/<pid>/status
+from this you should see something like
+task: 0f160000 ksp: 0f161de8 pt_regs: 0f161f68
+This now gives you a pointer to the task structure.
+Now make CC:="s390-gcc -g" kernel/sched.s
+To get the task_struct stabinfo.
+( task_struct is defined in include/linux/sched.h ).
+Now we want to look at
+task->active_mm->pgd
+on my machine the active_mm in the task structure stab is
+active_mm:(4,12),672,32
+its offset is 672/8=84=0x54
+the pgd member in the mm_struct stab is
+pgd:(4,6)=*(29,5),96,32
+so its offset is 96/8=12=0xc
+
+so we'll
+hexdump -s 0xf160054 /dev/mem | more
+i.e. task_struct+active_mm offset
+to look at the active_mm member
+f160054 0fee cc60 0019 e334 0000 0000 0000 0011
+hexdump -s 0x0feecc6c /dev/mem | more
+i.e. active_mm+pgd offset
+feecc6c 0f2c 0000 0000 0001 0000 0001 0000 0010
+we get something like
+now do 
+TR I R STD <pgd|0x7f> 0.7fffffff
+i.e. the 0x7f is added because the pgd only
+gives the page table origin & we need to set the low bits
+to the maximum possible segment table length.
+TR I R STD 0f2c007f 0.7fffffff
+on z/Architecture you'll probably need to do
+TR I R STD <pgd|0x7> 0.ffffffffffffffff
+to set the TableType to 0x1 & the Table length to 3.
 
 
 
@@ -1347,10 +1434,18 @@
 
 Help for displaying ascii textstrings
 -------------------------------------
-As textstrings are cannot be displayed in ASCII under the VM debugger ( I love EBDIC too ) I have 
-written this little program which will convert a command line of hex digits to ascii text
-which can be compiled under linux & you can copy the hex digits from your x3270 terminal to
-your xterm if you are debugging from a linuxbox.
+On the very latest VM Nucleus'es VM can now display ascii
+( thanks Neale for the hint ) by doing
+D TX<lowaddr>.<len>
+e.g.
+D TX0.100
+
+Alternatively
+=============
+Under older VM debuggers ( I love EBDIC too ) you can use this little program I wrote which
+will convert a command line of hex digits to ascii text which can be compiled under linux & 
+you can copy the hex digits from your x3270 terminal to your xterm if you are debugging
+from a linuxbox.
 
 This is quite useful when looking at a parameter passed in as a text string
 under VM ( unless you are good at decoding ASCII in your head ).
@@ -1532,8 +1627,8 @@
 
 
 
-s/390 & zSeries IO Overview
-===========================
+s/390 & z/Architecture IO Overview
+==================================
 
 I am not going to give a course in 390 IO architecture as this would take me quite a
 while & I'm no expert. Instead I'll give a 390 IO architecture summary for Dummies if you have 
@@ -1685,15 +1780,15 @@
 between 2 machines. We use 2 cables under linux to do a bi-directional serial link.
 
 
-Debugging IO on s/390 & zSeries under VM
-=========================================
+Debugging IO on s/390 & z/Architecture under VM
+===============================================
 
 Now we are ready to go on with IO tracing commands under VM
 
 A few self explanatory queries:
 Q OSA
 Q CTC
-Q DISK
+Q DISK ( This command is CMS specific )
 Q DASD
 
 
@@ -1998,9 +2093,14 @@
 This is done using a the same trick described for VM 
 p/x (*($sp+56))&0x7fffffff get the first backchain.
 
-For zSeries do
-p/x *($sp+112) i.e. replace 56 with 112 & ignore the &0x7fffffff
-in the macros below.
+For z/Architecture
+Replace 56 with 112 & ignore the &0x7fffffff
+in the macros below & do nasty casts to longs like the following
+as gdb unfortunately deals with printed arguments as ints which
+messes up everything.
+i.e. here is a 3rd backchain dereference
+p/x *(long *)(***(long ***)$sp+112)
+
 
 this outputs 
 $5 = 0x528f18 
@@ -2265,6 +2365,21 @@
 
 Some driver debugging techniques
 ================================
+debug feature
+-------------
+Some of our drivers now support a "debug feature" in
+/proc/s390dbf see s390dbf.txt in the linux/Documentation directory
+for more info.
+e.g. 
+to switch on the lcs "debug feature"
+echo 5 > /proc/s390dbf/lcs/level
+& then after the error occured.
+cat /proc/s390dbf/lcs/sprintf >/logfile
+the logfile now contains some information which may help
+tech support resolve a problem in the field.
+
+
+
 high level debugging network drivers
 ------------------------------------
 ifconfig is a quite useful command
@@ -2339,65 +2454,10 @@
 & type ? in the debugger for help.
 
 
-Debugging Drivers
-=================
-Some of our drivers now support a debug logging feature in
-/proc/s390dbf see s390dbf.txt in the linux/Documentation directory
-for more info.
-e.g. 
-to switch on lcs debugging
-echo 5 > /proc/s390dbf/lcs/level
-& then after the error occured.
-cat /proc/s390dbf/lcs/sprintf >/logfile
-the logfile now contains some information which may help
-tech support resolve a problem in the field.
-
-If you have VM look at the chapter Debugging IO on S390 under VM.
-
-
-
-
-Tools soon to be available
-==========================
-
-Dumptool & Lcrash
------------------
-Michael Holzheu & others here at IBM have a fairly mature port of 
-SGI's lcrash tool which allows one to look at kernel structures in a
-running kernel.
-
-It also complements a tool called dumptool which dumps all the kernels
-memory pages & registers to either a tape or a disk.
-This can be used by tech support or an ambitous end user do
-post mortem debugging of a machine like gdb core dumps.
-
-Going into how to use this tool in detail will be explained
-in other documentation supplied by IBM & the lcrash homepage
-http://oss.sgi.com/projects/lkcd/.
-
-How they work
--------------
-Lcrash is a perfectly normal application
-however it requires an additional file.
-It is built using a patch to the kernel source base.
-
-
-Debugging a live system it uses /dev/mem
-alternatively for post mortem debugging it uses the data 
-collected by dumptool.
-
-
-Ltrace
-------
-We also have a tool called ltrace in our CVS repository 
-no plans on a delivery date yet.
-ltrace is a superset of strace in that it also allows
-tracing of shared libraries calls as well as system calls,
-man ltrace for more info.
 
 SysRq
 =====
-This is now supported by linux for s/390 & zSeries.
+This is now supported by linux for s/390 & z/Architecture.
 To enable it do compile the kernel with 
 Kernel Hacking -> Magic SysRq Key Enabled
 echo "1" > /proc/sys/kernel/sysrq.
@@ -2426,14 +2486,14 @@
 Various info & man pages.
 CMS Help on tracing commands.
 Linux for s/390 Elf Application Binary Interface
-Linux for zSeries Elf Application Binary Interface ( Both Highly Recommended )
+Linux for z/Series Elf Application Binary Interface ( Both Highly Recommended )
 z/Architecture Principles of Operation SA22-7832-00
 Enterprise Systems Architecture/390 Reference Summary SA22-7209-01 & the
 Enterprise Systems Architecture/390 Principles of Operation SA22-7201-05
 
-
-
-
-
-
+Special Thanks
+==============
+Special thanks to Neale Ferguson who maintains a much
+prettier HTML version of this page at
+http://penguinvm.princeton.edu/notes.html#Debug390
 

FUNET's LINUX-ADM group, linux-adm@nic.funet.fi
TCL-scripts by Sam Shen (who was at: slshen@lbl.gov)