Here is the code I have so far in symbolstore.py

My portion is specifically lines 36 – 71 where I am now gathering up a string of source files along with the revision number, and generating a (almost) properly formatted stream file in the pdb’s directory.

Here are the barriers:

1. I would prefer to have a template where I have a variable %SOURCEFILES% that can be replaced and output to a new file for each pdb, instead of writing out each line of the stream file in symbolstore.py. If I can’t get that happening for this release – oh well, Dave said it can be a bit ugly at the 0.3 stage. At least what I have right now, is creating the stream file properly.

2. On lines 38 and 39 – what I am making here is a string that looks like this: c:\mozilla/xpcom/glue/nsTraceRefcnt.h and in fact, this has to be c:\USERS_SOURCEFILE_DIR\mozilla\xpcom\glue\nsTraceRecnt.h — so two things need to happen here, first, I need the / to be \ (except for the one after c:, cause it’s right already) and second, for now I can probably name the dir “ff” and the user can make sure that VStudio is pointing to that path (c:\ff) when looking for source files…but in the long run? I don’t know yet.

3. This is along the same lines as the template issue. The reason I haven’t successfully implemented the template is because I don’t know where to put, or how to create a path to, the right folders when symbolstore.py is being called. Unfortunately, it is not as easy as I had hoped in that putting the template in the same folder as symbolstore.py would allow me to open it with its name only. So as well as not being able to open the template, I don’t know how I can call upon pdbstr from inside of symbolstore.py

That’s going to be the last part, the first two are a little more pressing.

Huge changes since last night’s post.

I can insert a .stream file (see previous post for what a .stream file data block contains) into an unindexed .pdb file via pdbstr -w as long as it has certain variables.

I still need to test what the minimum info needed is, right now I just copied and pasted from one that works. I will slowly delete variables from it until it doesn’t work anymore.

Using cvs to pull a specific revision of a file looks like this: cvs export -r 1.23 filename

So – in my test.pdb.stream file I changed the CVS_EXTRACT_CMD to :

CVS_EXTRACT_CMD=%fnchdir%(%CVS_WORKINGDIR%)cvs.exe -d %fnvar%(%var2%) export -r %var4% -d %cvsdatetarg% %var3%

(p.s. %var#% are based on * delimited text in the source files section)

In the source files, i add a source file that contains cvs in the symbol file
like so:

SRCSRV: source files —————————————

Then, when I write that to the accessiblemarshal.pdb file followed by a call to srctool -f on that .pdb file, I am greeted with a source indexed file that contains the cvs command that the source server will use to extract the source code when the time comes.

The result looks like this:

[c:\ff\mozilla\accessible\public\msaa\AccessibleMarshal.def] cmd: cvs.exe -d :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot export -r 1.3 -d 12-01-07 mozilla/accessible/public/msaa/AccessibleMarshal.def

c:\ff\mozilla\objdir_debugInfo\dist\crashreporter-symbols\2007120119\accessiblemarshal.pdb\8EEE4D7316254E22AE3D99EB4082744C3\accessiblemarshal.pdb: 1 source files are indexed – 199 are not

So now I need to learn how best to add this into the symbolstore.py file.

My first instinct is to add something in the ProcessFile function that keeps track of all the files that start with cvs, they look like this:

FILE 2 cvs:cvs-mirror.mozilla.org/cvsroot:mozilla/accessible/public/msaa/AccessibleMarshal.def:1.3
FILE 3 c:/program files/microsoft visual studio 8/vc/platformsdk/include/RpcNsi.h

I already did a little playing around with the python script and was able to insert some code here where I wrote if filename.startswith(“cvs”) then some debug statement to see that I could in fact catch the right lines (which I did).

So let’s break it down:

  1. I need to track all lines that start with “cvs” and then turn these into proper file paths
  2. I need a (minimal) template .pdb.stream file that I can write the source file paths to in source files section
  3. Then I will call pdbstr -w and write that .pdb.stream (each named for the pdb it will write to) to the appropriate pdb file

The final hurdle – learn to write good python code to make this all happen.

Working on 0.3 release of source server is all about getting the pdb files gathered up when symbolstore.py is made and then feeding them to pdbstr.exe. I’ve copied the binary into the mozilla/toolkit/crashreporter/tools folder where symbolstore.py is called and I’ve spent the last 4 hours or so playing around with python.

So far I have learned this:

1. THIS IS WRONG, PLEASE IGNORE That I can call pdbstr.exe on an unindexed pdb file and it will in fact read and generate a data block that has all the information needed to set up the cvs cmd calls – see this output from calling pdbstr -r -p:thepdbfile -s:srcsrv > thepdbfile.stream ** for corrections see next post **

2. The next part would be to write this block back to the pdb file i think – so that it’s available to the source server

This is what calling pdbstr -r gets me:

SRCSRV: ini ------------------------------------------------
DATETIME=Tue Nov 27 03:56:53 2007
SRCSRV: variables ------------------------------------------
SRCSRV: source files ---------------------------------------
SRCSRV: end ------------------------------------------------

So the next goal is to successfully keep a list in symbolstore.py of all the pdb so that pdbstr can be called on them.

Okay, this is take 2 because my first post was unceremoniously ‘disappeared’.

My BTS team is getting geared up for writing Unit Tests in JUnit. We’re using Eclipse with subclipse for SVN. Two teammates are Windows, one Linux and me on the Mac. Mac’s version of Java is Apple’s not Sun’s so my JRE profile is not compatible with theirs. This means I will be using my Linux VM for the project – getting to know Ubuntu a little better.

Today’s the first day of significant coding and I am already frustrated that I have to reach back to use the ctrl key instead of my trusty apple key for shortcuts. Also the tilde key actually produces < and there’s no delete functionality (what Mac calls delete is really backspace).

So I have found a solution and here it is:

First, create a file (mine is called wickedCoolKeycodeFinder – you know, so I can always find it again)
and in that file put this:
xev | grep -A2 --line-buffered '^KeyRelease' | sed -n '/keycode /s/^.*keycode \([0-9]*\).* (.*, \(.*\)).*$/\1 \2/p'

chmod +x this file and then run it to produce a neat little window that will tell you what keycode is returned for whatever key you press.

Then, once you know the keycodes of the keys you want to map, create a file called .xmodmaprc and in that file I put:

!this sorts out the apple key + backspace to do delete
keycode 113 = BackSpace Terminate_Server Delete
! map tilde and grave
keycode 49 = grave asciitilde
!map apple key to ctrl
remove Control = Control_L
keycode 115 = Control_L
keycode 116 = Control_L
add Control = Control_L

Now I just need to figure out how I can get my two-finger scrolling on the touchpad to work and I’ll be a Linux user forever!

So for my 0.3 release of sourceServer, I am tasked with altering the code in symbolstore.py so that it will track the source files and then call pdbstr.exe on them.

To do this, I need to understand what PDBSTR.EXE does. This is what I have to go on:

Once the code has looked at all the source files in the PDB file, it’s time to call PDBSTR.EXE to write the index stream, called SRCSRV, to the PDB file. When debugging, the debugger looks for this stream. If it is found, the debugger knows there’s a source server involved and calls into SRCSRV.DLL to execute the version control system to ensure the right file is accessed.

I’ve been studying the pdb files pre and post indexing to gain insight into what happens when I use the Debugging for Windows srcsrv tools. Circling the symbolstore.py code getting ready to make changes to see what I can do.

Any minute now the big picture will come into focus…back to my pdb files.

Just a quick post to share my SourceServer┬áPresentation if anyone’s interested in an overview of what my project this term in DPS909 has entailed.

XPcom Lab II

Well there was an initial kerfuffle trying to get the extension “firstxpcomchrome” working in my Minefield build. Turns out that when I was attempting to connect it up via a text file in the profiles/myprofile/extensions folder, I had put the path as a c:/…. path which, in Unix (ie Mac), is not correct. This was affecting my ability to install the extension through drag n drop because the file for firstxpcomchrome@senecac.on.ca was not being overwritten so even though the extension appeared to install correctly in the add-ons manager window, upon restarting it was nowhere to be seen. Deleting the erroneous flat file and doing a drag and drop installation cleared this up and I was able to continue with the lab.

Aside from crashing the browser every time I called on “Reload All Chrome” from the Extension Developer add-on, the lab went smoothly.

(please enjoy the File menu behaviour that is happening in the background)