- how and what I need to reference

- what I should put in my design / analysis parts

- how and where to use UML diagrams.

At the moment I have finished to write the first draft of the abstract, introduction, analysis and design parts.

I am now going to work on the modifications we talked about during the meeting.

I also need to work on finding some more references.

I’ll then start working on the implementation part.

The big sections of my report will be :

- Introduction

- Analysis

-Design

-Implementation

- Feedback/ Testing

- Conclusion

We also discussed what each section should content.

I have now finished to write the abstract and introduction, so now I’m now going to start writing the Analysis and Design parts.

I also told him I had decided to stop coding on the project. Here are the main reasons :

- The final year project handbook says students are supposed to work 20 hours per week on their final year project (and 10hours per week for each of the other modules), over a period of 20 weeks. Which makes a total of 400 hours for the whole project. So far I’ve spent at least 800 hours on the project (report not included). Also I have four other modules, on which I should have worked 40hours per week but I didn’t.

- The marks : even if it was not my main objective until now, I need to remember that the objective of the year is to get a first class degree. And for that it makes much more sense to work on the other modules than on the final year project, for the simple reason that the implementation only represents 30% of the marks.

So from now on I will only work on the report and the other modules and won’t be changing the code any more.

My supervisor didn’t tell me if it was a good idea or not to stop the development, he just told me that I still had to keep a schedule for the remaining weeks.

Also, from now on we will only meet if needed (I won’t go every week as I used to).

Here is the schedule for next week : I’m going to start writing the abstract and the introduction parts of my report.

It didn’t really go according to the plan: I spent all thursday and friday fixing bugs and trying to loading new datasets in order to have some more interesting data to show during the presentation.

As a result I didn’t work at all on the VR part.

I now have several scenes : 3 using some simulated data, and 1 using some real data.

The presentation went well, about 10-15 people intended, and they all seemed impressed by the demonstration.

During the meeting we discussed about the future of the project :

- I won’t modify the code any more.

- They don’t really know how they want to make the library evolve. So, in the next few months they’re going to use it, and after they will start thinking about possible improvements or new features.

- Some people suggested to transfer the project to the Visualisation intern

- My project report will be the only developer documentation available.

- I’ve implemented a very basic VR mode : asymmetric frustum for the head tracking, stereo display, wand input to control the camera and the time. But this has not been tested yet so it’s hard to tell the current status of the VR part.

- I’ve implemented a graph view : basically you have a rectangle surface on which you can plot graphs, and animate it at the same time as the other objects in the scene. You can also zoom in / out on one axis or both.

- I’ve added a bit of shading on the BHA so that it looks less flat.

- I’ve created a binary serializer / loader for the data files (Float list and Double list), the only files which can’t be serialized as binary are the bha composition files, but it’s not a problem as they are really small (~10kB)

I’ve debugged the multi threading and the multi windows system : now you can create as many 3D popup windows as you want, and use different cameras in each of them. (I’ll post a video of that later)

I’ve added several functions to the C Interface. (I still need to document it though)

To do next :

- debug the borehole generation : still have some twists problems.

- add the new functions to the Matlab interface

- work on the documentation of both the interfaces.

- Improve a bit the GUI.

- Fly mode camera (for the VR)

I’ve done some research and here are a few useful links :

- This guy did a lot of work about stereographics, and he also put some OpenGL code :

http://local.wasp.uwa.edu.au/~pbourke/miscellaneous/stereographics/stereorender/

http://local.wasp.uwa.edu.au/~pbourke/miscellaneous/stereographics/ 

- I need to use Active Stereo, and the only way I know to do that is to use the Quadro FX driver from nVidia : http://www.nvidia.com/object/quadro_stereo_technology.html

Here is what I’ve found on the forum nvidia :

“This is correct. Stereo support is one of the selling points of the Quadros. The specific feature is called quad buffers. That is; left and right front and back buffers.
To use quad buffers, first select a pixel format with quad buffers/stereo support.
Then use glDrawBuffer(GL_BACK_LEFT/GL_BACK_RIGHT) to select the buffer. No openGL extensions or special API needed.”

http://developer.nvidia.com/forums/index.php?showtopic=3007

So, apparently I just need to set the “stereo” flag in my pixel format when the OpenGL context is created (which means using WX_GL_STEREO : http://docs.wxwidgets.org/2.8/wx_wxglcanvas.html)

and then I need to render twice the scene using glDrawBuffer(GL_BACK_LEFT) and glDrawBuffer(GL_BACK_RIGHT) , like this :

http://www.orthostereo.com/geometryopengl.html  

GLvoid display(GLvoid)
{
  glDrawBuffer(GL_BACK);                                   //draw into both back buffers
  glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);      //clear color and depth buffers

  glDrawBuffer(GL_BACK_LEFT);                              //draw into back left buffer
  glMatrixMode(GL_PROJECTION);
  glLoadIdentity();                                        //reset projection matrix
  glFrustum(leftCam.leftfrustum, leftCam.rightfrustum,     //set left view frustum
            leftCam.bottomfrustum, leftCam.topfrustum,
            nearZ, farZ);
  glTranslatef(leftCam.modeltranslation, 0.0, 0.0);        //translate to cancel parallax
  glMatrixMode(GL_MODELVIEW);
  glLoadIdentity);
  glPushMatrix();
  {
    glTranslatef(0.0, 0.0, depthZ);                        //translate to screenplane
    drawscene();
  }
  glPopMatrix();

  glDrawBuffer(GL_BACK_RIGHT);                             //draw into back right buffer
  glMatrixMode(GL_PROJECTION);
  glLoadIdentity();                                        //reset projection matrix
  glFrustum(rightCam.leftfrustum, rightCam.rightfrustum,   //set left view frustum
            rightCam.bottomfrustum, rightCam.topfrustum,
            nearZ, farZ);
  glTranslatef(rightCam.modeltranslation, 0.0, 0.0);       //translate to cancel parallax
  glMatrixMode(GL_MODELVIEW);
  glLoadIdentity);

  glPushMatrix();
  {
    glTranslatef(0.0, 0.0, depthZ);                        //translate to screenplane
    drawscene();
  }
  glPopMatrix();
  
  glutSwapBuffers();
}

I don’t remember what kind of grpahics card we have in the lab, so I’m not sure I’ll be able to test it here. I need to check that.

The first step is to be able to read the data from the tracking system.

In the past I’ve used VRJuggler , this is an open source framework dedicated to Virtual Reality. It’s really flexible (this is the most flexible library I’ve ever used), and it’s supporting the Intersense IS900 tracking system (which is what Schlumberger has in its iCenter).

As I said I knew this API before I started the project, but I decided not to use it to handle the graphics part because it can’t be link statically and so it would have implied a lot of DLL to distribute with the library which is exactly what I wanted to avoid.

Compilation

So, first I had to recompile VRJuggler because there are no binaries for Visual Studio 2008, this took me ages to do because there are a lot of projects inside the solution and a lot of paths to set.

Fortunately, I’ve found on internet a link to a large zip file containing all the dependencies  (compiled with VS 2008) for the library (375 MB !), so thanks a lot to the person who did that, I would have probably not had the time to do it myself.

Both Debug and Release versions compiled fine, but for some reasons only the Release version is working, the debug version crashes at initialization. I decided not to waste time on making the Debug version work because it’s not really important.

Environment variables

The main drawback of VRJuggler is that it depends on a lot of environment variables for the path. The reason is the library is so flexible that it requires to load a lot of config files at runtime. I really wanted to avoid using environment variables because it would have meant installing VRJuggler on the client machine, which is not really a good option.

After looking into the API code, I managed to find how to force it to use relative paths for the different files it needed. So now there is no need to install the whole VRJuggler package ! (I still need to test it on another computer to be 100% sure).

Config files

VRJuggler loads the hardware configuration from one or several “jconf” files at runtime (These are XML files), and there is a tool called VRJConfig to help generating them.And here is the second biggest drawback of VRJuggler : this is so flexible that the config files are quite complicated to generate, even with VRJConfig… (you need to set the drivers, the hardware, to create some proxys, windows, aliases,…)

And VRJConfig is a bit buggy too: it won’t save the configuration properly all the time, sometimes it writes, sometimes it doesn’t…

So, after a few hours spent on the configuration ( don’t forget I’ve already done once during my placement, I let you imagine what it was like the first time…), I managed to generate 2 configuration files :

- “intersense.jconf” : this is the config file to read the tracking system data directly from the COM port. (You can find here the 3 different ways to get IS900 data, basically you can : read from the COM port using VRJuggler driver, read from the COM Port using the Intersense API,  read the data from the Intersense Trackd Server). I decided to use VRJuggler’s driver.

- “intersenseSimu.jconf” : this is a config file which simulates an IS900 device and so I can change the device values using the keyboard.

VRJuggler Kernel

VRJuggler is using a system of kernel, this means there is no “main” function, instead we create a class which inherits from one of VRJuggler kernel class and we overload the methods we want.

At the initialization VRJuggler will start the kernel in a new thread.

I’ve written a new kernel based on the “Simple Input” example provided with the SDK and on the work I’ve done during my placement.

TrackingDLL

I’ve now started to write a DLL, it has a C interface. This way it can dynamically be loaded by the main API without having to recompile.

The interface allows the user to :

- start / stop the tracking system

- get the data for the 2 position devices (head / wand), 2 analogic buttons (joystick x and y axis), 6 digital buttons (red,green,blue, yellow, trigger, joystick).

- According to the data some short components (0.01 meters) have an orientation different of 90 degrees with the previous/next component, and their diameter is so big compared to their length (0.15  meters) that it should be physically impossible to fit. Here is a screenshot of what I’m talking about : (I’ve removed the liaisons between the cylinders so that you can better see the components) :

And here is a screenshot when I use my algorithm to reduce the overlapping between the cylinders (as you can see it’s still not enough to solve the collision problems) :

- Second problem : I know some components are meant to be deformed, but in this case it seems weird : the size of the component is multiplied by 10 (usually it’s 0.1 and it’s extended to 1), and also all the components seem to extend when they arrive at the same point.

…

9,1,-122.70001239,105.40006676,2223.00000000,0    ==> Time 9, there is more than 1 meter between the nodes 1 and 2
9,2,-122.70001286,105.40007063,2221.99999931,0

Here are two screenshots and a video to explain :

In this video you can see that each component arriving to the same point will be extended in the same way :

instead of calling “init()” at the beginning, the user can call “initNewThread()”, which will launch the API and start refreshing the window in a secondary thread.

After that the user can load the trajectory, borehole, bha etc… from the main thread, and still use the window at the same time.

Also, the user can programmatically control the window( Like setting the time for example) without having to handle the refresh loop (like it was before).

Finally, I’ve added a progress bar and an “abort” button in order to follow the loading process and be able to cancel if it’s too long.

Problems encountered

My classes were not thread safe (the mutex were not used correctly, for example I was locking a resource,returning a reference, unlocking the resource, and then using the reference.) So now this is all fixed. (I’ve added explicit functions to lock /unlock the data)

OpenGL calls can only be made from the thread in which the OpenGL context has been created, 2 solutions to fix this problem :

- create a second context in the second thread and share the resources: this solution was not easy to implement with my design (the opengl context is managed by wx, so I should have created a new window, and also it means I should check before each openGL call that the right context is active).

- send messages to the thread which can use opengl so that it makes the call for the other thread : this is the solution I’ve implemented; at the beginning of each “update” loop the secondary thread check if there are some opengl requests coming from the other thread. In the meantime the other thread is blocked and wait for the other thread answer.

So far it’s working fine like this.

There are two problems left :

- when I close the application the two threads don’t seem to synchronize correctly.

- the shaders don’t seem to work for the moment.

I’ll check that tomorrow.

- The trajectory : just by increasing the “SegmentSize” property of the curve the bug disappears.

- The borehole : I’ve added a property called “DistanceDirection”. This property uses an averaged direction instead of the real direction when the points are too close.

For example, DistanceDirection = 10.f : if the distance between the p0 and p1 is > 10 => we keep the vector p0p1 as the direction.

the distance between p0 and p1 <10, and if the distance between p0 and p2 <10 too, but the distance between p0 and p3 > 10, then we take the average of p0p1, p0p2, p0p3 as the direction.

Here is the screenshot of the old method :

And here is the new method result :

- The BHA : With the previous dataset all the BHA components were almost aligned so I hadn’t this problem, but now there are some steep angles between two consecutive cylinders, which means there are collisions :

I wrote an algorithm to move cylinders out of each other, without changing the direction of the components :

“P” is the common point to both the cylinders,

V0 is the direction vector from P to the other extremity of the first cylinder.

V1 is the direction vector from P to the other extremity of the second cylinder.

V2 is the normal of the plane containing V0,V1 and P ==> V2 = cross(V0,V1)

V00 is a vector normal to V0 in this plane, its length is the radius of the cylinder 0 : V00 = cross(V0,V2) * Radius_0

V11 is a vector normal to V1 in this plane, its length is the radius of the cylinder 1 : V11 = cross(V1,V2) * Radius_1

Now we want to move each cylinder along its axis of the same distance so that we only have one point of contact left :

t * V0 + V00 = t * V1 + V11

t –  t * ( V1 / V0 = (V11 – V00) / V0

t = (V11 –  V00) /  ( V0 – V1)

We now move the points of each cylinder along their respective axis of “t” times the cylinder direction

Here is the result :

Problems left :

There are still a lot of bugs on the BHA, I think it’s because of the data : some cylinders are very short, and at some points during the drilling operation they are compressed, as a result I get some cylinders like this :

This is impossible to use this to compute the orientation of the component.

This is an accuracy problem, so I can’t really see any solution to that.

In this scene the BHA is oscillating a lot more than in the previous one, also the Borehole has a stairs-like shape.

As a result there are some big visual bugs :

- the cylinders of the borehole seem to collide with each other.

- The trajectory interpolation doesn’t work at some points.

- the BHA deformation doesn’t work properly.

Looks like I’ll have a lot of debugging to do in the next few days !

This week we talked about the wrapper/interfaces the library should have.

C++ :

pros : this is the native language of the API, so it’s the most flexible interface.

cons: the client application has to be linked to the library with exactly the same compiler (Visual Studio 2008) or it won’t work, it also requires a lot of headers.

C:

pros: C can be loaded from almost any other language, as Schlumberger wants to be able to load the API from fortran it seems to be a good choice. Also the DLL is not compiler dependant.

Finally, there is only one header to distribute and a DLL.

cons:

We can only use structures and functions so we won’t have access to all the features of the API.

All the functions have to be loaded from the dll using “GetProcAddress”

Can’t handle exceptions.

.NET:

pros: object orientated, we have access to all the API features. It’s compatible with any other .NET language (most of Schlumberger softwares are written in .NET).

cons: it requires the .NET framework at runtime.

Matlab: This is using the C interface.

So I decided to implement a C interface and Matlab wrapper for the moment. Later I’ll try to also write a .NET interface.

Virtual reality

Apparently the university virtual reality room is only for research, students don’t have access to it.

I’m going to use VRJuggler for the VR part, and I know you can create some dummy devices to emulate the wand and head tracking, so I’ll do that for the beginning. And later I’ll ask the new visualization intern at Schlumberger to do some tests for me in the iCenter.

It also works fine on the labs machines and on my housemates’ computers.

This week I’ve been trying to contact different people from the School of Science and Engineering in order to see if they could let me use Matlab on one of their machine, but without any success : nobody replied to me.

I’ve also tried to contact Jean Luc Lugrin, to have more details about how it works to book the virtual reality room, and have more details about the equipment available. He didn’t reply neither yet.

So this morning I went in person to the school of science, somebody told me to go to the ICT helpdesk in the Stephenson Building, and there they told me there was Matlab in all their labs, and that I could use any of them.

I went to one of the lab, to check if I could log in, and everything is fine. 

I’ve started to do some research about how to call C++ DLLs from Matlab, here are a few links :

http://www.mathworks.com/support/tech-notes/1600/1605.html#C_warning

http://osiris.tuwien.ac.at/~wgarn/programming/DLLs_for_Matlab.html

http://www.mathworks.com/support/tech-notes/1600/1605.html

I need to look into it in more details, but I have the impression I just need to write an interface which will call my C++ code. As my library tries to avoid using objects on the client side, it shouldn’t be too hard to do.

Next step is to work on the VR part if I can get in touch with Jean Luc, otherwise I’ll start the matlab interface.

- The library is now statically linked to the CRT libraries. ( some explanations about the different libraries can be found here) 

The main problem I had was coming from std::string : if I exported it in the Library, it wouldn’t link correctly in the application. 

So now there is not any public string attribute left in the classes. And for some reason I can only pass Strings from the main application to the library by reference (if I pass them by copy I get a memory heap corruption). 

For example in the TextOutput class I had forgotten a ‘&’ and it was crashing each time I was calling this function. I have no idea why this was happening, and it has been really hard to isolate and fix this bug, because the TextOutput class is used by all the classes of the API so every small change made to this class requires to recompile the whole API ( > 10 minutes…) 

- The demonstration applications were crashing on some machine when the user was closing them : 

This bug has been particularly hard to track too, because it was not happening on my machine. 

I’ve looked on internet, and some people were suggesting to generate the linker map file and to find to which instruction the address was corresponding. Unfortunately I didn’t manage to find the correct address, so instead I  reduced the number of lines called in a test program until I was able to isolate the bug. 

And apparently the problem was coming from the fact that I was deleting the OpenGL buffers at the end of the execution. 

I’ve checked the OpenGL documentation :  http://www.opengl.org/sdk/docs/man/xhtml/glDeleteBuffers.xml

and they don’t say anywhere that I’m not supposed to free the buffers at the end. On the other hand since I don’t remove the buffers anymore, I don’t have any memory leak, and no more error, but I’ll still have to check later why it’s doing that.

- I can now have several OpenGL windows at the same time, having different contexts but sharing the same resources.

When I was thinking that I had a problem with my depth buffer on the child window, actually it was a problem of initialization of the projection matrix. (the near and far planes were not initialized…), which is why the depth buffer was not working as expected and also the shaders didn’t seem to work.

I have recorded a new video demonstrating the two windows system, I’ll post it later.

The BHA is one single mesh made of cylinders of different diameters.

The positionning and deformation are done in a shader.

All the information is passed using a 2D float texture : each row represents a different time.

On one row there is for each node the angle(1 float), the position (3 floats)  and the colour value(1 float).

The time is passed as a uniform variable.

Time Control

I’ve created a “Time” resource in the CoreManager, everything in the API which is using time is using this variable.

This means by controlling this single variable, it’s possible to control any animation which occurs in the API.

Here is the interface of the TimeController :

float operator()(); //return time
void play();
void playBackwards();
void stop();
void increaseSpeed();
void decreaseSpeed();
void nextFrame();
void previousFrame();
void setSpeed(const float);
void setDeltaFrame(const float);
void setDeltaSpeed(const float);
void setTimeLimits(float min,float max);
void disableTimeLimits();

And I’ve created a Wx toolbox, which basically just calls the above methods :

We talked about the best way to create chilren windows which share the same OpenGL resources that the main window.

So to sum up :

There is :

- a device context per window

- a rendering context.

There are two ways to share resources between two windows :

1) at each window update we link/unlink the device context of the window which needs to be refreshed to the rendering context.

=> According to my supervisor this technique is slowing down the application a lot.

2) each window has its own rendering context, and we tell the rendering contexts to share their resources.

=> That’s what I need to do.

In a first time, I’ll try to create several windows in a standalone application, once it will be working properly I’ll try to integrate it in Wx.

This week, I also need to find who to contact in order to have access to Matlab, and to the VR Cave.

I’m also going to finish the implementation of the new BHA modelling and animation.

This one was mainly centered on the 4D surface and the BHA positionning inside the borehole.

4D Surface

There is no big change to do to it, they are happy with the way it works at the moment.

There has been one suggestions :  they would like to be able to split the surface in several sections along the depth axis, so that in the case of a trajectory rotating on itself they don’t lose any information.

BHA Positionning

We discussed for a while to decide wether or not we should use procedural shapes instead of obj models.

problems with obj files :

- too many vertices on one circle of the cylinder shape, and there might not be enough on the length.

- the stabilizer (blue shape on the following drawing), is made out of two small cylinders with a bigger one in between, the length and diameter of each of this cylinder may vary. This makes it impossible to scale using only  x,y and z scaling factors.

On the other hand, using procedural shapes fix these problems, but we will lose the small details on the components. (it’s a purely aesthetic problem).

Procedural shapes specifications

Instead of having a list of components to map onto a skeleton, we directly have a list of bones which are defined by a length and an outside diameter. Each bone is identified by a name.

In the previous drawing, the nodes represent the articulation between two bones. There are three reasons for passing from one bone to another :

- a change of diameter (nodes 2 and 3)

- a change of component (nodes 1,4 and 6)

- a deformation of the component  (node 5)

This will be described in a CSV file, and from now on it will be called a “BHA file”.

Example of CSV file:

ELEMENT,LEN,OD
n,m,m
1,0.09700000,0.21460000
2,0.03600000,0.21460000

The second file will contain the offset information for each component at each time.

It will be called “BHA position File” or “deformation file”

“TVD”  “NS” and “EW” now represent an absolute position (instead of an offset)

Note: the user can provide a position file without a time column to show the BHA in one position only.

The last file is called a “BHA colour file”, it looks like a calliper file: 1 (optional) column for the time, 1 column “Node”, followed by a number of columns of colour data.

For example if there are 20 colour data columns, it means one column represents 18 degrees.

GUI

They confirmed that they were not interested in having a big GUI. They’re really only interested in the API side. So I’m going to remove it from the milestones and keep it for the very end.

Test on Schlumberger’s machines

I’ve run the demo applications on several employees’ laptops (new and old laptops), and it worked fine on all of them : no slow downs,  no big memory or CPU usage.

Which means, I can keep the shader solution for the deformation.

Why several windows ?

with several windows you can :

- visualize different data on the same models. For example one window will map the ratio current angle / max angle onto the BHA, and the second window will map the current offset.

- monitor several things at the same time. For example you can follow the drilling bit in one window, and display a graph in the child window.

Problems

However I still have two bugs :

- There is no depth test in the children windows.

- The children windows don’t have access to the shaders.

Todo list

For the moment it’s not possible to link the same camera to two windows at the same time.For example, you move the camera in one of the window and the camera is automatically moved in the second one as well.

Following the last BHA Mapping post :

- I’ve implemented a shader version of the BHA Mapping :

The shader is using two uniforms:

- the size of the texture used to pass the parameters

- a texture sampler

All the data is passed through the texture.

the first 6 values represent the x,y and z scaling factors, the number of articulations in the component skeleton, the maximum angle of deformation, and the maximum vector offset.

Then for each articulation we have :

- the z distance from the front of the component

- the angle of deformation

- an offset vector.

Note: the maximum angle and maximum offset are here to compute the colour value of the vertex, they may vary depending on what we want to represent: we might want to visualize the value of the current deformation angle over the maximum angle of this component, or it could be over the maximum angle of the entire BHA.

Then the shader is :

- scaling the model

- moving the vertices according to the offset vector

- rotating the vertices according to the deformation angle.

- transforming and shading each vertex.

Results

It’s working fine:

- almost no pre processing required

- no duplication of data

- the user can change the information mapped onto the BHA during the execution.

- the shader is quite small, so it’s not too expensive to run. For the moment I’m not doing the light calculation, so I’m skipping the normal transformation.

Animation

All the parameters are passed using one row of the texture, the obvious way of implementing animation is to use one row=one time.

This will be the next step.

Problems

For the moment I don’t have the models for the BHA components. The only one I have is the drilling bit. So I’m using cylinders for the other components.

The good thing about using procedural shapes is that I have the exact tesselation I need: not too many vertices on one circle, and more vertices on the length so that I can deform the cylinder accurately.

I’ve made a test and used the drilling bit to represent all the models: it was really slow, even though I was drawing only 1000 ft of bha in a 15000 ft well.

Ideas of solutions

I need to implement a visibility test: all the components are never all visible at the same time. http://www.lighthouse3d.com/opengl/viewfrustum/

I need to check with Maurice if we could use procedural shapes instead of the obj files.

http://zach.in.tu-clausthal.de/teaching/cg2_07/literatur/glsl_tutorial/index.html

Several tutorials from gamedev.net:

http://wiki.gamedev.net/index.php/OpenGL:Tutorials:Loading_and_using_GLSL_shaders

http://wiki.gamedev.net/index.php/OpenGL_shader_variable_types

- A component library : this is a collection of 3D models identified by a string. All the components are centred in 0,0,0 and orientated toward +Z. In order to achieve that, when a model is first loaded (from an ASCII obj file), it’s manually orientated/positioned by the user; then it’s saved as binary. The rest of the time the application will only use the binary model.

- A Skeleton file: this is a csv file (see previous post)

- A Composition file: this is a csv file contains 3 columns : name of the element (it has to be present in the library), length of the element, outside diameter of the element.

The list is ordered from bottom to top : this means the first element is the one situated at the bottom of the borehole.

Note : an element can be used several times, each time with a different length and outside diameter.

The user has to pass an extra parameter : the current depth of the borehole.

Skeleton generation

1) find the 3d position of the current depth of the borehole.

2) From there I move back along the trajectory of the depth given in the skeleton file.

3) I offset this point of the given vector and associate the rotation value to this point. Note : the rotation value is a torsion angle representing the deformation of the component.

Components mapping

For each component of the list:

- I make a copy of the model data present in the library

- I scale the model so that it matches the outside diameter and the length given in the composition file.

- I create a skeleton for the component : it contains a minimum of 2 points, the front and the back of the component. If there are some points of the BHA skeleton in between : I add them.

So now for each component I have a map like this :

map[0.0]  => Offset Vector (0,0,0) and torsion angle (10)

map[5.0]  => Offset Vector (2,3,1) and torsion angle (8)

map[60.0]  => Offset Vector (1,2,0) and torsion angle (6)

map[90.0]  => Offset Vector (-1,1,0.5) and torsion angle (5)

Contrary to what I said in my previous post, the components are mapped like this on the skeleton :

And not like this :

The offset vector stored in the map is relative to the normal position of the point (contrary to the BHA skeleton one which is absolute) :

Component deformation : CPU vs GPU

So now we have all the data needed, the only question is : where to do the deformation and the shading ?

Here are the advantages of using a shader :

- no pre processing : it’s perfect for real time

- no data duplicated : the model is loaded once, the scale and shading are done on the GPU. (A lot of memory saved)

- the user can change the shader in real time. To visualize where the component is the most bent, where the whole bha is the most bent, to visualize the torsion, etc…

Drawbacks :

- Requires OpenGL 2.0, I tried to check what it meant, and from what I understand from this article, this means the graphics card has to be at least a geforce 3 (this model has been released at the beginning of 2001). Schlumberger is using several 3D visualization software, that’s why I’m sure their machines have real graphics cards and not just an intel graphics chipset.

- Expensive at each frame (and as the scaling is non uniform, we need to renormalize the normal, which implies a square root)

- We can’t read back the values.

- If we want to remain compatible with a maximum of graphics cards, the shader shouldn’t use more than 44 uniform floats. To run the shader needs the length of the mesh, the number of control points, and the maximum deformation,  then for each point of the component skeleton we have : the length relative to the front of the component, an offset vector(3 floats) and an angle.

This means 3 variables + a maximum of 8 control points for the skeleton.(5 floats per point)

Note : we could use a texture to store this data, in which case the space is not a problem anymore, but the shader will require more resources to go and read the texture.

For the moment I’m going to implement the shader version using a texture for the input data. I’m going back to Schlumberger next thursday so when I’ll be there, I’ll try to run my application on several of their machines to check what kind of hardware they have.

I will have a library of BHA elements, each element will have a name length and outside diameter size.

For the moment I have only the model for the drilling bit, so I will use some cylinder for the other pieces.

Positions

So basically I have a skeleton which is defined by a list of joints.

Each joint is defined by :

- a depth offset: this is the distance along the trajectory from the current bottom of the borehole.

- a vector offset : which represents the distance from the trajectory.

- an angle of rotation. The axis of rotation is the trajectory direction.

Note : The articulation points don’t necessarily correspond to the BHA components.

How to compute the components positions

If we assume the angles between the joints of the skeleton are all > 90 degrees ( I need to check with Maurice), then we have this :

The black line represents the skeleton

B and C are some joints of the skeleton

A and X are the extermities of one BHA component of length d. We know A and we need to find X position.

We know the distance between X and A, it means for the moment all the potential X are on the sphere of center A and radius d:

We also know that X belongs to AC, which means we can express X like this:

If we put these two equations together and develop, we get :

We now have a quadratic equation:

Which can be solved like this:

Note : n has to be between 0 and 1, that’s why we only keep the positive solution.

The general algorithm to extrude a surface out of a curve is pretty simple:

We duplicate and translate each point of the curve in the extrusion direction :

(the red points represent the original trajectory, the blue points the first extrusion and the green ones the second)

This surface will be used to plot some data which is borehole-depth and time related: the trajectory represents the borehole-depth dimension, and the extrusion represents the time.

Then for each time and depth the data is represented by a colour and/or a vertex displacement up or down along the normal (not implemented yet).

For the moment we have one piece of data for each combination of depth and time, but in reality it doesn’t make sense to have some data displayed for a section of the borehole which has not been drilled yet.

So in 2D, with the horizontal axis representing time and the vertical one depth, we would have :

In blue : this is the surface as it’s represented at the moment (data for all the time / depth combinations)

In red : this is what it should look like (data only for the existing time / depth combinations) 

So for each depth I remove all the vertices which have a time inferior to the time of drilling of this depth. Then I move the first vertex of the row to the exact time:

(you can see the first vertex of the two last rows has been moved from its original position.)

The second problem is: how to link the vertices correctly when you don’t have the same number of vertices on the two rows?

The rows are generated from top to bottom (which means row 0 = top row)

Let’s say there are n vertices on the top row and n-m vertices on the second. with m >=0 (it’s impossible to have more vertices than on the previous row : it would mean the time is going backwards).

The indices are generated in two steps:

1st step: the vertices are between 0 and m : we draw only the top triangles of the mesh

2nd step: the vertices >= m : we draw the top and bottom triangles of the quad.

This algorithm gives this :

Here is the result I get by putting everything together :

Note :

- I don’t have any data to plot on the surface yet : I need to generate some fake one.

- I’ve generated the drilling times myself (that’s why you can see some big edges)

What’s next ?

I still need to do the vertices displacement up and down on the surface, I also need to do the surface manipulation so that it can translate and intersect with the trajectory, but for that I need to implement a picking system (which I don’t have at the moment neither).

I was also thinking about implementing a surface in 2D (like in the second diagram), dispay it in a secondary window and put a vertical moving line which would represent the time and which would move at the same time as the drilling bit.

But for the moment, my priority is to implement the BHA inside the borehole, Maurice sent me some more explanations about the format of data I will have in input, so I can now start to implement this part.

We also talked about the order in which I should implement the features left.

Presentation

click here to see the presentation

The presentation included explanations about :

- the API architecture, and how it makes it possible to replace any part of it.

- how to extend the library by using the interfaces present in the API.

- how the Parameters class works, and why it is more flexible and safe than normal parameters.

I then made a sum up of the features implemented and which ones still need to be implemented.

I finished the presentation by showing 3 small demonstration applications.

Feedbacks

They all seemed satisfied with the presentation and demonstration.

New ideas

They would like to be able to map the points of contact between the BHA and the borehole on the borehole.

It should already be possible to do in a “static” state as it’s already possible to map any kind of picture on the borehole.

The main problem is going to be how to animate the texture.

What’s next ?

It’s be decided that the two more important things left are the visualization of the BHA inside the borehole and the 4D surface.

They’re not particularly interested in a very complete GUI, so its development has been moved to the very end.

To implement the BHA visualization I need some data files which I don’t have at the moment, so instead I’m going to start working on the 4D surface.

I’ve also added a Connector class, which allows ObservedData to be connected between themselves, which means by using the following few lines of code I now have a drilling tool rotating on itself and going to down the borehole, and a trajectory camera following it:

//move the object to its bounding box centre and orientate it so that +Z is the front

Vector3f min = Drive.drawables["testObj"]->getBoundingBox().getMinimum();

Vector3f max = Drive.drawables["testObj"]->getBoundingBox().getMaximum();

cml::matrix_rotation_axis_angle(rot,Vector3f(1.f,0.f,0.f),- PI / 2.f);

cml::matrix_translation(transfo,

(min +

max) * -0.5f);

//Controller which makes the tool rotating on itself

RotationController* rotate = new RotationController;

rotate->axis = Vector3f(0.f,0.f,1.f);

rotate->speed = 10.f;

rotate->inMatrix = rot * transfo;

//Controller to move along the borehole

FollowBoreholeController* follow = new FollowBoreholeController((BoreholeDataObject*)Drive.dataObjects["borehole"]);

follow->speed = 150.f;

//connector to combine both the rotation of the tool and its position along the borehole

Drive.addConnector(new Connector<Matrix44f>(&rotate->outMatrix,&follow->inMatrix));

//connect the output of the controller to the tool transformation

Drive.addConnector(new Connector<Matrix44f>(&follow->outMatrix,&Drive.drawables["testObj"]->parameters.getData<Matrix44f>(“Transformation”)));

//connect the same controller to the camera

Drive.addConnector(new Connector<float>(&follow->distanceFromStart,&Drive.cameras.getActiveCamera()->data.getData<float>(“DistanceFromStart”)));

Drive.controllers["rotateObj"] = rotate;

Drive.controllers["followObj"] = follow;

I’ve also implemented a small compass, which sticks to the top right corner of the window.:

But it doesn’t work fine yet, and it looks pretty crap, so I’m not sure if I’m going to keep it or not yet.

For the moment I’m starting to work on the trackball camera, I’ve found a page which explains very well how to implement one : click here

He found a bug with the texture: you can see a line going along the cylinder where it’s been generated (on the left on the picture)

The problem is coming from the linear interpolation done by the texture manager: it interpolates the first row with the last one, the first column with the last one etc… which doesn’t work in this case because it’s not a cyclic texture. To fix that I stopped using the first and last row and column to store data, and I’ve duplicated the first and last element of each row, and the first and last element of each column so that the linear interpolation keeps working fine.

But this didn’t entirely fixed the problem because there is also a problem in the way I generate the texture. In the following picture I’ve just copied/pasted the texture next to itself, and you can see there are some artefacts:

I’ll check the algorithm tomorrow.

Today I’ve also added a new category of classes : the Controllers, they will be used to animate the Drawables objects. As the animation is not the main goal of this API I decided to create this category instead of implementing a complete scene graph.

For the moment I’ve only created a “rotationController” that I use to make the drilling tool rotate on itself.

I’ve also added a “type” field in the Data class for both ObservedData and normal Data. So now the user can request for a specific object what are the parameters it accepts:

To do that I used typeid().name().

Unfortunately, each time the “name()” function is called, the returned string is detected as a memory leak (even if actually it’s not one).

This bug exists for at least 3 years, Microsoft has a page about it here, and the only “solution” they propose is “The reported memory leak can be ignored.” (Great ! Thank you for the advice !Take your time to fix the bug…)

So I’ll have to live with it I guess, even though it’s really annoying because I can’t tell if there are some real memory leaks hidden in the middle of all the fake ones.

I have pretty big radius variations:

But I’ve also noticed some big variations in the log as well, so maybe it’s normal. (I’ll ask confirmation)

9603,8.5,8.5,8.5,8.5,8.5,8.5,8.5,8.5,8.5,8.5,8.5,8.5,8.5,8.5,8.5,8.5
9603.5,8.6821,8.6457,8.6092,8.5727,8.5363,9.3479,10.1596,10.9712,11.7829,10.9763,10.1697,9.3631,8.5565,8.5879,8.6193,8.6507

Without the look up table:

With a 1000 samples look up table:

I’ll talk about it with my supervisor today

Yesterday I have migrated the minimum curvature interpolation method : it works fine, it’s a lot easier to implement than a 3d points interpolation, and it’s linear !

I’ve also implemented the diameter loaded from file(with a different radius per angle)

There are a lot of bumps on the borehole, I need to check with Maurice, because they seem to be present in the data too.

And to finish: the prettiest bug of the day!

I got only one problem during its implementation : obj indices start from 1 and not 0.

Fortunately for me, Mathieu my housemate had already implemented an obj parser, so when I told him I had a bug he knew the problem was probably coming from this (and he was right!).

For the moment I have only one model (and the material file is missing that’s why it’s white but Maurice told me he would try to get it for me):

OBJ files are easy but quite long to parse (it’s all in ascii)

For example for this model (1526 KB) it takes :

7.88 seconds in debug

1.46 seconds in release

So I’ve used the boost serializer to save it as binary (464 KB) and now the parsing is a lot faster:

0.62 seconds in debug

0.038 seconds in release

For each circle of the borehole, the “0″ of the x coordinates has to be at the top, which means I have to find a normal to the trajectory always pointing to the same direction.

The first algorithm I tried was:

dir = borehole direction

up = (0,1,0)

right (1,0,0)

x = cross(dir,up);

y = cross(dir,x);

for each angle :

point = center + x* sin(angle) + y*cos(angle)

This works fine most of the time except when the direction is close from being vertical.

In this case Maurice told me the northest point of the borehole was considered to be the top:

So I generate the vectors like this:

if(fabs(dotProduct) > 0.9)

{

dir = borehole direction

up = (0,1,0)

right (1,0,0)

y = cross(right,dir);

x = cross(y,dir);

for each angle :

point = center + x* sin(angle) + y*cos(angle)

}

Now the texture coordinates are correct but there is a twist at the point where we switch from “north” to “up”:

To avoid that I’ve added a step in the algorithm described above:

if the dot product is between 0.92 and 0.90 : I do a linear interpolation in the direction the trajectory is starting to point at

float delta = 1.f – ((dotValue – 0.9f) / 0.02f);

float angle=0.f;

if(fabs(dir[0]) > fabs(dir[2]))

{

if(dir[0] > 0.f) angle = PI/2.f;

else angle = -PI/2.f;

}

else

{

if(dir[2] < 0.f)

{

if(dir[0] > 0.f) angle = -PI;

else angle = PI;

}

}

matrix_rotation_axis_angle(rotation,dir,angle*delta);

The transition is not really smooth :

But in practice, I’m going to map seismic data on the pipe, and in this case it looks ok :

Seismic data texture generation

The seismic data have the following format :

Depth angle_0 angle_1 … angle_n

The number of columns may vary, so to compute the angle that each column represents I do 360/ (nbColumns -1)

Then for each vertex of each circle of the borehole I convert it into a depth + angle, then I do a tri-linear interpolation with the data I have.

Texture coordinates :

I generate the textures coordinates this way :

1 pixel on the height = 1 depth of one borehole circle.

Then if there are more control points than the texture height, I split into several columns (cf one of the previous post).

The user can pick the texture size (as long as it’s a power of 2), so if he takes a bigger one, the column will be larger, and so it will be more accurate. By default, for compatibility reasons the texture size is set to 1024.

Also I duplicate the first and last pixel of each row of each column: this is because OpenGL is using them when it’s doing a linear interpolation of the border. (source)

So in the end I generate a texture which looks like this:

Note : for the moment I have only part of a log file, that’s why most of the values are “unknown”, so for debugging purposes I’ve duplicated the part I have:

I tried different interpolation methods :

- Lagrange Cubic : didn’t fix the problem

- Cosine interpolation : fix the problem, but as it’s taking only two control points there is no continuity.

- BSpline interpolation : doesn’t pass by the control points

- Hermite Cubic interpolation: didn’t work with default parameters, but by tweaking the tension and bias parameters it now works !

 (source)

Links:

Different interpolation methods

Cubic Hermite

Lagrange polynomial

Trigonometric interpolation

Cubic BSpline

I kept the interpolation methods which didn’t work in the API because they might work just fine for some other curves.

The second problem I’ve fixed is the uniform tesselation: as the interpolation method is not linear, I need to convert the interpolant variable from what I want to the real one.

For example for the Cubic Hermite interpolation, I’ve created an array of 100 samples using the control points (-1,0,0,0),(0,0,0,0),(1,0,0,0),(2,0,0,0) with 0<=t <= 1

Then each time I want a certain interpolant value, I do a look up in my samples map to find the closest values and I do a linear interpolation between them.

I’ve also almost finished the camera which follows the trajectory, it just needs some more testing and a parameter to pick the speed.

I’m now going to work on the seismic texture generation.

(0,0,0),( 702.0997,0,0), (774.2777,-0.2254,-0.0593) with 1<= t < 2

( 702.0997,0,0), (774.2777,-0.2254,-0.0593) (833.3321,-0.4927,-0.1495) with 0 <= t< 1

I think I understand where the problem comes from : there is a big x variation in the first segment and no y variation (that’s why the curve is not smooth) and also the x variation of the second segment is a lot smaller so the linear interpolation is a lot more influenced by the first segment than the second one, that’s why the interpolated points pass the control points and then go back to it.

If I’m right, I don’t see any way to fix the problem, so tomorrow I’m going to change the interpolation method and use a quadratic lagrange interpolation instead of 2 cubic + a linear interpolation.

[Edit:] Sorry I have inversed the names: the interpolation I’m using at the moment is quadratic (3 control points) and I’m going to move to cubic (4 control points)

Today I’ve added some basic shading in order to make the borehole look more 3D, I’ve fixed the aspect ratio problem, the borehole can now be rendered with texture and transparency at the same time (by rendering the back part of the pipe, inversing the culling and rendering the second part). I’ve also started to implement an orbital camera which moves along the trajectory.

Once I will have finished the camera I’ll post some more video/pictures.

Links :

face culling

opengl lights and shadows

materials and lights

help about setting up a directional light

In order to fix that I’ve duplicated the first vertex changing its texture coordinates like this:

The second problem is quite similar, I store the seismic data in a texture like this:

so between 1023 and 1024 the y textures coordinates needs to go from 0 to 1, and the x ones has to be incremented by the stride value.

Same method to fix the problem : I duplicate the last circle and change the texture coordinates on it:

He fixed my aspect ratio problem :

- the field of view angle was too big

- the window was not updating the aspect ratio until it was resized.

He suggested me a new way to render the borehole so that I have the transparency working with textures:

- I render the back of the pipe (using the culling)

- then I render the front part of it (by inversing the culling)

Also he suggested to add some lighting effect so we see more the cylinder shape.

Things I have to do for next week:

- Fix/improve the camera system : I should move to a trackball camera system. For that I need to find some documentation on it.(I remember there is one implemented in OpenSceneGraph so I’ll probably start by having a look at osg’s code).

- Borehole semi transparent rendering with textures + lighting.

- generate seismic texture.

- implement a camera which follows the trajectory

- obj files loading

- updating use cases (I need to put only actions in them)

Links : textures/blending tutorials (code sampler, pyopengl, man glblendfunc, glBlendFunc tutorial)

Here is the problem :

Note : the half of the borehole near the camera is blue, the other one is black.

On the first picture the transparency is working fine : the triangles are rendered from back to front. ( Final colour = (buffer combined with the back face of the pipe) combined with the front face of the pipe)

On the second picture I’ve just rotated a bit the camera and a problem appears : the front triangles are rendered first which means now : Final colour = (buffer combined with the front face) ERASED BY (buffer combined with the back face of the pipe) ==> the front face is now ignored.

I’ve tried to find some technics, but it seems that semi transparency is still one of the big problem of computer graphics, some progress has been made concerning depth peeling but it still is at an early stage.

Depth peeling allows us to draw translucent objects in any order, draw intersecting translucent objects, is pixel-level accurate and can be implemented on fairly old hardware. The problem with it is that it’s slow. For every translucent layer that could ever conceivably overlap there needs to be an additional complete geometry pass

(source)

http://www.opengl3.org/wiki/Alpha_Blending

http://www.realtimerendering.com/blog/hpg-2009-a-closer-look/

However: the problem disappears if I’m not using a texture (as both of the side of the cylinder has the same colour, it doesn’t matter which one is rendered first).

So to sum up :

- transparency : only if there is no texture.

- Interactive cutting plane otherwise.

Next thing will be to try to implement the cutting plane. I got a good idea for that: if I render a cylinder and set the culling so that it’s the outside triangles which are culled, then the graphics card will do all the work for me !

There are a list of functions available to do “low level” things, and there are some macro functions (the ones in blue) which regroup a list of calls to the low level ones.

 The red boxes represent the dependencies to other objects.

(click on the picture to see it in full size)

I still have 2 problems though :

- the aspect ratio is always wrong and I can’t find out why… (I’ll ask my supervisor)

- Depth problem / rendering semi transparent objects : in order to render semi transparent objects the depth test has to be disabled which is why the lines of the back face of the bounding box are in front of the curve. However I’ve found several ideas about how to fix that : (gamedev forum) (gamedev forum 2) (opengl forum).

From what I’ve read real semi transparency is pretty hard and expensive to achieve. However using a few tricks we can get some pretty good results.

The easiest one to implement is to render all the solid objects first and then the semi transparent ones. ==> That’s what I’m going to try.

In case the result is still not good enough, I could then sort the semi transparent objects from back to front (using their bounding boxes).

We’ll see the result of the first technic, but I think the result should be good enough for what we need here : the Borehole is the only object of the scene which is semi transparent, so if I render it last most of the problems should be solved. The only one left would be when you can see two parts of the borehole overlapping (like in the screenshot).

Next : I’m going to try to fix the problem described above and add the texture coordinates to the borehole (it shouldn’t be too hard)

I will also ask Maurice some seismic data to map onto the borehole. I still need to do my user cases scenarios.

Here are the data I have to draw it :

- a trajectory

- a list of  [distance from start/diameter] couples

To represent the borehole I’m going to draw some kind of cylinders (except that the diameter will be different at each extremity of them).

For that I’m going to need a normal to the curve for each point. To compute it I’m going to need the tangent to the curve (I’m going to take the direction from one extremity to the other of the cylinder for that), then I need to take an arbitrary vector which is different from the tangent one, and from the cross product of these two vectors I’ll get my normal.

The previous orbital camera I had implemented was using a pre translation, a quaternion and another translation.

Each frame I was combining the new quaternion with the previous one, and because of the float rounding some inaccuracy was starting to appear after a few frames (especially when the camera was moving a lot).

My supervisor recommended me to use polar coordinates (site1   , site 2 and site3 ) instead of a quaternion: I implemented it today and the main problem with the polar coordinates is when you’re approaching the poles .

When you’re rotating the model from left to right: you’re expecting it to rotate around the current camera up axis.

 When you’re at the “equator” level, it does that, but when you’re getting to close from one of the pole it starts rotating around the camera direction axis instead of the up one.

2nd problem : To update the up vector, I compute the new camera position, then I compute the new camera direction, then I compute the rotation to pass from the original direction to the new one, and finally I apply it to the original up vector. And this technic doesn’t work when the two directions gets almost parallel and opposite. (It does the same as when you’re trying to rotate the object and the camera is near a pole).

To avoid these two problems I’ve added some intermediate variables, now I have :

original position, temp position, current position. (same for up,right and direction)

Temp position represent the origina of the current system

Temp up the y axis of the current system, temp right the x axis and temp direction the z axis.

At the start originalPosition = tempPosition.

then I keep combining the polar coordinates but I keep the same temp vectors frame after frame until theta or phi is approaching from PI.

(PI for phi means it’s getting close from a pole, and PI for theta means it’s going to start lining up with the original direction)

At this point the current position/up/right/direction become the new origin/axis of the system, I set theta and phi back to PI/2 (which is the “middle” of the equator)

This method works perfectly :

- Combining angles is a lot less dangerous than combining quaternions (there are less rounding)

- I don’t need to combine 3 different matrices anymore : now I have an interest point, a distance from it and two angles.

Controls

I moved to more intuitive controls :

arrow keys up & down to zoom in/out

arrow keys left & right to move the interest point to the left/right.

mouse wheel to zoom in/out

maintain left click and move the mouse up/down left/right to rotate around the interest point.

Moving Scale

In the 3D software I’ve used before, the main problem to move in the 3D space is that the “delta move” is hardcoded which means : scrolling for ages if your scene is pretty big, or miss all your objects if it’s small.

To avoid that I decided to use a system of ratio instead of an absolute value : when you zoom in the distance between you and the object is reduced by10%, if you zoom out, it’s increased by 10%, if you want to move the interest point to the right/left, it will move it of 10% of the distance between the camera and the interest point,etc…

For the moment it’s working pretty well : I take the center of the scene bounding box as interest point, and I position the camera so that it can see the whole box, then the user can move around the scene as described above.

I guess the main problem in the future will be to determine what is the point of interest the user is looking at.

Next : I’m going to start working on modelising the borehole.

- More UML diagrams (user cases)

- Justify the technic used to slow down the application (and try to find a better one if possible)

- Orbital camera implementation

- Borehole representation

For the moment I’m keeping CML, I’ve found out how to disable the static asserts, so now I can export my template classes. I’m in touch with the developpers of the library (see this topic), I explained them the problem, and they are going to check to see if it’s safe or not to remove these static asserts.

wxWidgets :

For the moment I still can use only a console application, but I’m following two threads (here and here) on wxWidgets in order to fix that. 

Mutex:

I have a design problem : at the moment I can’t store thread safe variables inside STL containers because these containers require a valid copy constructor, however I can’t lock an object if it is constant (as the mutex value needs to be changed). So two solutions to this problem :

- I can create an unsafe accessor, which won’t use any mutex. This solution would probably work, but it creates a weakness in the variable.

- I could remove the “constant” flag from the object using const_cast  in order to lock it. (I didn’t try it yet, but I’m not sure it will work as in the link it says the variable still shouldn’t be modified)

Anyway, for the moment I don’t really need to use STL containers to store thread safe variables so I will try to find a solution later if I really need to use them.

Mouse + Keyboard events:

I’ve implemented an interface to capture keyboard and mouse events. (page of wxWidgets wiki about WxGLCanvas)

Models:

Today I talked to Maurice, and he provided one model of tool, it’s in OBJ format, so I’ll probably need to write a parser for it (but from what I know this file format is pretty easy to parse so it shouldn’t take long).

Maurice also told me to not waste too much time on the GUI : scientists are more intereted in the features of the API than in the graphics interface.  So I should probably let the GUI for the end if I have some spare time. (and see with my supervisor if I can change my specifications).

This lib had some pretty good advantages :

- all the classes are template, which means no DLL or .lib

- it can switch from row to column vectors without changing the syntax in the code.

Unfortunately CML is using static template asserts in its code, which makes it impossible to export in a DLL.

So, to fix this problem there are several solutions :

- edit CML code and remove all the asserts : it’s not a good idea, because CML really rely on these asserts to check the code validity. Removing them could make the library really unstable.

- do a wrapper for the vectors and matrices like I did for the STL containers : it’s a lot of work, and it will slow down the API.

- change of maths library.

I don’t want to use a huge maths library, I just need to do some linear algebra. So far Eigen seems to be the most suitable one. I didn’t try it yet though, but the syntax seems really clear. The only drawback is it doesn’t seem to handle row/column vectors.

This class is a simple thread safe DataManager : each piece of data is identified by a string.
its interface allows to :

- register a new piece of data
- get a piece of data by specifying its id string.
- check if an ID string has been registered.
DataManager::Data interface :
- get the value
- set the value
Accessing an element in a map which uses strings as ID can be pretty slow, so in order to speed things up, it’s possible to get a reference directly to the SecureObject.

Example without the reference :

[for each update]

uint widthScreen = DataManager::database().getData<uint>(“WidthScreen”);
/* Update aspect ratio here */
 
Example with the reference :

[During the initialization]
DataManager::Data<uint>& widthScreenRef= DataManager::database().getData<uint>(“WidthScreen”);
 
[For each update ]
uint widthScreen = widthScreenRef.getValue();
 

EventManager:

This class implements a pretty intuitive thread safe event/callback system, the interface allows to :
- register a new event
- get an event reference
- trigger an event
- add a callback to an existing event
- check if an event exists.
 EventManager::Event interface :
 - trigger event
 - add callback
 
Here is an example of how to use it :

/* register an event which will transmit a “bool” value to the callbacks (in this case the boolean won’t mean anything)*/
eventManager.registerNewEvent<bool>(“onRender”);

In the rendering loop :
onRenderEvent.trigger(true);

To add a callback :
class RenderTriangle:public BaseClass
{
 public:
  /* the callback function has to accept the same type of parameter as the event it’s connected to*/
  void render(const bool&)
  {
   /*draw the triangle here*/
  }
};

[in the code]
RenderTriangle r;
/* Note : you need to pass a pointer to an object and to a method of this object */
/* Also : the class of the callback HAS TO inherit from “BaseClass” */
eventManager.addCallback(“onRender”,&r,&RenderTriangle::render);

ObservedDataManager
This class is just a combination of the two previous ones: it stores data, and trigger an event when a piece of data is modified.
here is its interface:
- register a new piece of data
- get a piece of data by specifying its id string.
- check if an ID string has been registered.
ObservedDataManager::Data
- get value
- set value
- add callback

For the moment DataManager and ObservedDataManager are singletons because I wanted them to be available from anywhere in the code, but I’m going to change that because it should be possible for each module to have its own Data Storage which should be independant from the CoreApplication one. So instead of doing a singleton, the CoreManager will contain an instance of both of the Data managers and as the CoreManager is a singleton this means the Data will still be available from anywere in the API.
 

Note : I’m still hesitating to keep these 3 classes or just keep the ObservedDataManager as it contains the features of both the other classes, but on the other hand it implies a small overhead if you want to use it only to trigger events, or as a simple data manager.

Class Diagrams

Communication diagrams

Use case diagrams

For the moment I’m going to mainly use some commented class diagrams to explain my design, and some communication diagrams to describe the way it’s executed at runtime.

From the specifications : “the API has to be flexible “, that’s why I decided to split it into modules which will communicate with each other using some interfaces. This way all the modules are independant from each other.

But how to split into modules ?

The most obvious way is : one external library = one module. This way if one day we decide to replace a library by another one the only thing to do is to reimplement the interface with the new library. For the moment I’m using two external libraries : one for the GUI (wxWidgets) and one for the 3D (OpenGL).

Another good thing to do is to group all the platform specific functions in one module. It’s the same reason here : if one day we decide to make the API compatible with another platform we’ll only need to reimplement the interface for the new platform.

In order to make the API easier to use for the scientists, I decided to make the core application as a singleton : this way they won’t have to handle any object, and it’s also to be sure there is only one instance of the API used at a time. Also the API has to be useable from the GUI, so the fact that the core is a singleton means the GUI can make API calls the same way the user would do programmatically.

So here is the class diagram representing what I just explained :

We now need a way to initialize the core without it knowing which concrete classes are used.  For that I created a factory interface (ICoreFactory), which is responsible for creating a GUI, a 3DEngine and a platform object.

Then a concrete class implementing this interface has to be created for each combination of GUI/3D Engine/Platform.

For the moment this factory is passed as an argument to the CoreManager “initialize()” method :

Two things to notice :

- The factory is responsible for creating the 3 objects which means if we decide to change one of the three objects we need to recreate a factory for the three (which is not normal), this is because I can’t separate the 3D engine creation from the GUI creation because I need to know which 3D Engine is used in order to initialize wxWidgets properly (it’s not the same sort of Canvas for OpenGL or Direct3D for example), but I’ll try to fix that soon ( I think I know how to avoid that)

- The second thing is the syntax used :  Drive.initialize(new WxGraphicsInterface); which is not really clear on wether or not the Core is going to delete the object, and also it’s contradictory with my “no need to handle objects” policy. Mathieu (my housemate) pointed me out a way to avoid this which is described in “Modern C++ design”, I’ll explain it more in details in a next post but basically it consists in creating the factories as singletons which register to the CoreApplication in their constructor. This way we can call them using an ID instead of a pointer.

- Doing a wrapper for the STL containers seems to be the best way to export them..

- I need to do more diagrams (use cases,etc) to justify my design.

- In order to VBO to work, the machine needs to be compatible with OpenGL 2, as my application will probably use only static drawing, my supervisor suggested to use displaylists instead (which makes the application compatible OpenGL 1.1)

Objectives for next week : I need to do the diagrams, keep on migrating things from my prototype solution, start working on the well surface representation.

Then I created the functions to map this interface data types to OpenGL ones.

Finally I have created a generic interface to manage vertex and index buffers, and to draw meshes using VBO  (The opengl documentation also helped me)

I also got again problems with the DLL warnings, it’s really anoying as each time I’m using a new instance of a template in my dll I need to export it (if I don’t it will crash when the client will call the function without necesseraly saying from where the error comes from…). It also means I have to put  export lines at different places in my code (I don’t think it’s possible to group them all in one place) which makes my code less readable.

I’m using glew to load openGL extensions and I had a bit of trouble to initialize it because its header has to be included before any opengl header, and it has to be initialized after opengl which was a bit hard to do as opengl initialization is done by wxGlCanvas and I was not too sure at which point it was done.

Next Step : migrating the curve interpolation functions, and the camera system.

Some explanations :

- The classes in the left column (DataManager,etc) are all singletons : they can be accessed from anywhere in the API

- An ICoreFactory object is passed to the “CoreManager” class, it’s in charge of creating the platform, graphics interface, 3d engine which will be then used by the CoreManager.

- CoreManager is for the moment the only interface of the API (see below for a code sample)

- MenuBar is a singleton for the moment, but it’s not necessary, so I’m just going to put it inside the AppWx class and provide an interface “add/remove menu” through the EventManager.

Code Sample (How to load the trajectory data from a text file):

#include <Drive/CsvTrajectoryParser.h>
#include <Drive/CoreManager.h>
#include <Drive/Types.h>
#include <Drive/Exception.h>
#include <iostream>

using namespace drive;

void main()
{
        try
        {
                Drive.parsers.registerTrajectoryParser(“csv”,new drive::CsvTrajectoryParser());

                ParametersMap param;
                FloatArray floatArray;
                Drive.parsers.loadTrajectoryFile(“data/Cupiagua_surveys.csv”,floatArray,param);
                //param is a map<String,String> it now contains some information concerning the file:
                //Number of points / Number of components per point / unit for each column / name for each column
                
                while(Drive.update())
                {
                        //main loop
                }
        }
        catch(Exception& e)
        {
                std::cout<<”EXCEPTION :”<< e.what()<<std::endl;
                system(“pause”);
        }

}

Note :   it’s just an example, in the future the csv parser will automatically be registered, and it will be possible to do something like LoadAndDisplayTrajectory(filename) without having to deal with the param and floatArray.

- this doesn’t concern private objects which can never be accessed from outside their parent class. ==> in these cases the warning can be disabled because there is no risk of corruption  (how to disable a warning locally)

- in the other cases the classes have to be exported. In the case of std::string I managed to export it :

However for std::vector and std::map they both depend of too many template classes, so I preferred to do a wrapper for both of them (source Note: in the source they say the article is obsolete for the versions of Visual Studio superior to 5.0, but as I got the problem described in the article I’m not too sure what it means)

Today I’ve also added the “Parser Manager” to the API, and I’ve implemented the csv parser to load the trajectories.

Next thing to do : integrate the curve generation and the camera system from the previous solution. I also need to create a first UML graph in order to have a more global view of my current design

I have lots (~50) of both these warnings when I compile. From what I understand it’s because I export some classes which depend from classes which are not exported.

The problem is I can’t export template classes, I can only export instances of them. So according to this link, I should export all the different instances of template I’m using.

The cause was not a wxAUI bug, but more a weird design choice : the docking event is triggered by the “onIdle”  evebt (I would have thought it would be triggered by ”onMouseUp”…) .

And as I was manually handling the event loop I was never sending “onIdle” events…

So for the moment I’m trying to recompile by myself the wxAui library and to link it directly to my project in order to debug it myself, but another problem is that the only Visual Studio solution provided is a VS6…

I’ll also fill a bug report on the wxwidgets tracker.

Boost multi threading / mutex : http://www.ddj.com/cpp/184401518?pgno=3

Dllimport /Dllexport : http://wiki.tcl.tk/8721

Detecting memory leaks : http://msdn.microsoft.com/en-us/library/w2fhc9a3(VS.71).aspx

Help on the wxAuiManager (panel manager) : http://www.twinforms.com/forum/viewtopic.php?id=69 and http://docs.wxwidgets.org/stable/wx_wxauimanager.html

Now I have a working wxWidgets window, with a “PanelManager” which is used to register some floating toolboxes, and to display/hide them.

Now I need to integrate OpenGL in the wx window. But I need to separate as much as possible OpenGL from wx, in case in the future OpenGL would be replaced by another rendering engine, or wx would be replaced by another GUI.

OpenGL integration : http://wiki.wxwidgets.org/WxGLCanvas

DirectX integration : http://www.gamedev.net/community/forums/topic.asp?topic_id=500976 (just used as a reference to see the differences)

Also, concerning my last post: I managed to improve the error given by visual studio when I forget the ‘&’ sign, now it’s possible to get a meaningful error :

ImpossibleToCopy<SecureObject> c;
 /*      You are not allowed to create a copy of this object !
  *      Are you sure you didn’t want to create a reference
  *      and you forgot the ‘&’ ?
  */

And also it indicates the line of the user code which causes this error.

To fix that, I decided to limit the refresh rate to the screen frequency. (I found how to detect the screen frequency : Here )

 So far I’ve created an EventManager class,a DataManager class and a ObservedDataManager class.

EventManager:

 This is an event library, basically the modules can register some events in the application, or register some callbacks to some existing ones.

 Each time one event is triggered all the corresponding callbacks are called.

 Each event is identified by a string, but it’s possible to store a reference to the event to speed up the access.

DataManager:

This class handle all kind of data, and make it available to all the modules.

Each piece of data is identified by a string, and it’s also possible to store a reference to the data to speed up the access.

ObservedDataManager:

This class is a fusion of both of the previous ones : some piece of data is stored, it’s possible to access using its string id, but an event is also triggered each time its value is modified.

These three classes are thread safe, and they are all singletons, but it’s also possible to create another instance of them locally.

I’ve created them as singletons so they can be accessed from anywhere in the API. (not to limit there number). With this design it’s like having all the data as global, which can be dangerous as any module can have access to it, but on the other hand it keeps the API very flexible. When I was on placement, it happened quite ofter that I was stuck to implement something because the API didn’t let me access the data I needed at this place of the API. I could improve the security by limiting the number of classes which can trigger an event or modify a data and let read access to everybody, but I’m not sure yet if it’s worth it.

Also I hesitate to keep both DataManager and ObservedDataManager classes, because they are very similar, but the Observed version implied an overhead (an eventManager is created for each piece of data, and an event is triggered each time the value changes).

The last thing is not a technical problem, it’s just I don’t like the syntax to access the data/events: as I want to prevent the user from deleting registered events, I have to return a reference and make the constructor/destructor private.

 ObservedDataManager::Data<uint>& refreshRate = ObservedDataManager::database().getData<uint>(“ApplicationRefreshRate”);

But if I forget to create “refreshRate” as a reference, it won’t compile as there is no constructor available (which is probably a good thing), but the error message produced is really obscure:

Note: it doesn’t even say from which of my file/line the error comes from.

The second problem with references is that they have to be initialised at the construction and can’t be changed later (at least I didn’t find how)

Instead of references I could use pointers: I can modify them at any moment,but the problem with pointers is that the overloaded operators become useless.

Two threads : One to display (GUI + OpenGL) and a secondary one to do all the computations (File loading, interpolation,etc). Like this, even if the application is performing intensive computations, the user remains in control: the UI doesn’t freeze, he can cancel the computations if it takes too much time for example.)

The two threads are synchronized using an event system: each thread can register some events and some callbacks.

So far I’ve created a template Event system.

Now I need to make it thread safe. I’m probably going to use the Boost library for the threads and mutex management.

Today I also compiled WxWidgets for windows (there is no existing binaries), I got a lot of trouble to do it because there is no existing solution for Visual Studio 2008 and of course the existing solutions can’t be converted.

So I had to edit an existing batch files, and tweak it so it uses the VS2008 compiler. After several tries I finally managed to get the static debug and the DLL release libraries generated (not the other ones so far…)

Another problem I got is that WxWidgets is using a single Macro to register the window/initialize the api/ initialize the window/ run the application, and it doesn’t return until the window is closed.Also the macro is generating automatically the “main” function of the application, this system is quite handy when you’re writing a window application but definitely not when you’re writing an API: I can’t let the GUI handled the main function, because it would means it is aware of the whole architecture of the API and to do the communication between the modules, and it would also mean it would be almost impossible to change the GUI used in the future.

After a lot of tests and research on internent I finally managed to create a wxWindow from a console application and to manage the message queue of the window by myself (which means the window is no longer blocking a thread until the end of its execution). The problem is I can’t make it work in a window based application (without the console).

However, after some more research, I managed to make it work as a DLL library, which is perfect as this will be the final type of the API.

Now I’m going to work on the mutex, and start to implement the design I’ve described. I also need to find out how I’m going to store the different kind of data in the shared memory.

Windows XP

 Mac OS    

   GTK (linux)

Also, my API will be used by very different people, some of them will be comfortable with programming and will probably want a very minimalist window, and some others will prefer to use menus instead of typing commands, which means it should be easy to show/hide toolbars, and for that WxWidgets is perfect as it’s using a system of floating toolboxes that you can show/hide, resize, dock or separate from the main window.

Here is a small demo application I have written to show the floating toolboxes :

he suggested that I :

- change my “ObjectCamera” so it uses polar coordinates

- start to use more intuitive commands to move in the world (instead of the “FPS Keys” I’m using now, which are not intuitive at all)

- get rid of GLUT now, and start to directly use the GUI.

- also implement a zoom which only requires to click instead of drawing a rectangle.

- display some information about the selected point

- should be able to also select a point which is not a control point.

Bugs to fix :

- The aspect ratio is wrong.

- Camera zoom problem.

- The application is using too much CPU (It’s because GLUT is refreshing the screen all the time, once I will have moved to a real GUI it shouldn’t happen anymore.)

What I’m going to do for next week :

- change the camera system. (It should also fix my zoom problem) and the commands to move in the world.
- choose a GUI Library, explain why and start to use it.

I also need to contact Maurice to know :

- if he wants to be able to change the colour of the curve and/or which colour they are generally using.

- if the control points should be displayed or not. (if yes at which size)

- which information he wants to be displayed about the selected point. (distance from the top, 3D coordinated,etc)

- what are the commands used in the other software he is using (and if he thinks they could be improved and how)

- a First Person one : you can go forward / backward, right/ left, up/down and rotate the view with the mouse.

- an object manipulation one (I don’t know what is the name of this kind of camera) : you can move the object forward/backward, right/left and rotate it with the mouse.

I also tried to implement a zoom : basically my idea was to draw a rectangle on the screen with the mouse and to zoom in to that region.

For the moment it doesn’t work at all, I think I understand why but I don’t really see how to fix the problem.

Here is the idea :

we move forward the area selected on the screen until that area is as big as the current near plan. I won’t detail all the computations, but roughly here is what I do :

• I compute the distance between the actual near plan and the new one
• using some trigonometry I compute the vector along which I should move the camera in order to have the camera centered in the middle of the new near plan.

Here are the problems:

with xRec the width in pixels of the rectangle drawn

If I take xResolution = 1200, dNearPlan = 1, and I draw a rectangle of 100px on the screen : it will make move the camera forward of only 16 whereas the scale of the world is a lot bigger.

Also there is a second problem : if I tweak the near plan so that the camera moves over a bigger distance, it won’t move in the correct direction because the direction computation is not accurate enough (there are only ‘xResolution’ values possible for the ‘x’ of the direction, which is definitely not enough to move over a big distance.

The ideas I got to try to fix the problem :

- to tweak the near plan => I tried it, and it doesn’t work because of the direction inaccuracy.

- to tweak the field of view angle => it will improve a bit the direction accuracy but not enough.

- instead of drawing rectangle, the selection could be done by only clicking on an object.

Problems : how to find the object on which the user clicked :

- using the Z-buffer ==> not accurate enough (only 65535 values)

- using bounding boxes around the objects and doing a ray / box intersection : there is still the problem of the direction inaccuracy.

Solution : I could click /create a rectangle, and generate a pyramid from that, find the objects inside this pyramid and zooming in near them. It would fix the inaccuracy problem.

I’m now going to try to implement that.

the list of coordinates in the raw files is as  (MD, INC , AZI) with

MD : length of the arc of curve from previous point

INC : angle between direction of the wellbore and the vertical

AZI: angle between direction of the wellbore and north when the wellbore is projected on a horizontal plane.

So it’s easy to do adaptive tesselation on this type of curve as MD represents the length of the arc between two points.

I only used the minimum curvature equation that Maurice gave me to interpolate this type of curve.

The second type of input data I have is a list of 3D points (TVD,NS,EW) :

TVD: vertical depth (0 is on the surface, positive numbers go down).

NS: direction towards north (negative means towards south)

EW: direction towards east (negative means towards west).

I tried different methods to interpolate these points :  (source for the methods)

- cosine interpolation : 

      Advantages:

• it’s easy to compute.
• requires only 2 control points to interpolate.

Drawbacks:

• Uses a “cosine” which is time expensive.
• When there are three aligned points, the curve is not straight.

- cubic interpolation:

Advantages:

• continuity

Drawbacks:

• requires 4 control points : which is a problem for the first and last segment of the curve.

- quadratic Lagrange interpolation + linear interpolation:

Advantages:

• continuity
• only 3 control points

Drawbacks:

•  there is no easy formula to compute the arc length.

Lagrange linear interpolation: in order to have a smoother curve, between pi and pi+1, for every step I do the linear interpolation between the lagrange curve defined by pi-1,pi,pi+1 and the one defined by pi,pi+1,pi+2.

Adaptive Tesselation : (source)

Until now, I was dividing every distance between two control points by the same number of samples. A better way of tesselating is to choose the number of samples to use according to the length of the curve to interpolate.

In order to do that, I compute an approximation of the arc length of the curve by putting sample points along the curve and doing the sum of the segment between them.

Then I can compute the number of samples needed in order to have a regular tesselation :

- list of 3D positions ==> I used a cosine interpolation for this one.

- list of coordinates as (distance from origin,inclination,azimuth) ==> I used a minimum curvature equation (that Maurice provided me) to interpolate this one.

When I draw the two curves and the list of points in 3D, they almost entirely overlap which means both of the interpolations are working fine.

I also implemented a very basic first person camera in order to navigate in the 3D world.

I am now going to test different method to interpolate in between the points.

Once it will be done I’ll generate the curve in 3D this time

I have added the methodology I’m going to use and the milestones.

I am now going to start working on loading the trajectory files and plotting the points in openGL

They are using two ways to represent a trajectory :
- a list of 3D points.
- a starting point and then a combination of inclination(angle to the vertical), azimuth and distance from the previous point. (They usually use a minimum curvature to then interpolate)

Maurice told me to only work on the 3D points version for the moment.

One of the file also provides for each point the length along the borehole.

TODO: plot the 3D points and interpolate in between (I’m thinking about using a Bezier curve for that).

- I am not experienced enough to create a good design from the first try.
- I will need some feedbacks from Schlumberger people during the implementation. And I can’t expect them to be always available for me and give immediate feedbacks all the time.

So, my idea was to work in parallel on the design of the application (reading books, and structuring the main application), and at the same time implementing several small standalone applications. The two main benefits of this method are:
- I don’t need to wait for a Schlumberger feedback to work on something else.
- For each standalone application I will see what is needed to make it work (data, functions,etc) and then correct/improve my design accordingly.

- To represent cuttings inside the borehole using a 3D DLA (diffusion-limited aggregation) algorithm. The problem is it is going to overlap with my Procedural Visual Effects module. So I decided to not include it in my final year project and to implement a 2D one for the PVE module.

- To do a volume renderer to visualise the seismic data of the drilling area. It would be interesting from a visualisation point of view, but wouldn’t be really useful for Schlumberger. (They would be able to provide me some seismic data though).

My supervisor told me to correct different things in my first draft (I’ll post an updated version soon).

I also need to write some more categories : “Personal objective”, “Methodology” and “Evaluation method”.

- I tried to keep the same presentation/structure as for a module ICA brief.
- I also tried to merge together the Schlumberger specifications and my suggestions.

- describe every step of the development

- publish the links to the papers I have found interesting

- sum up the meetings I have with my supervisor