XREC
Results of XREC Crystal Centering on Test Database                                

Centering protein crystals on the beamline is an important stage in automated high throughput crystallographic data collection. XREC has been developed at EMBL to perform automated crystal centering. There is a need to assess the performance of the systems on a extensive experimental dataset both as guide to users on performance and to investigate ways in which both systems may be combined in the future. The image dataset currently consists of 321 image sequences of which:

Grenoble 250, Daresbury 20, Maxlab 4, APS 49, Hamburg 2

Detailed experimental crystal centering results on a 321 crystal image sequence dataset

Crystal Centering Process

Image sequence obtained at different goniometer rotations. Crystals centers are determined in image sequence using image analysis techniques. Crystal centers fitted to the (linear) equation of a circle using least squares technique. Goniometer rotated to centre crystal on the beamline

Image Annotation

The results given on the test database are calculated mainly for 2D images. It is not possible to accuarately create a 3D model of the crystal shape from the angular sequence of 2D images and therefore the annoatation of the crystal boundary coordiantes is performed using 2D images. To produce a single centering score for the image sequence two approaches have been used; to average results over image sequence and for point coordinates projection into 3D space (i.e. 3D distance). 
loop annotation explained
A typical experiment contains a number of images with a sequence at different angular positions some of these images will be imaged face-on and others at an oblique angle. It is desireable to annotate and centre on all images in the sequence to accuractly predict the centre of rotation of the crystal on the goniostat. There is more uncertainty involved in detecting and annotating crystal locations at oblique angles. This is due to a number of problems including
Ideally the crystal centre position will be within the boundary of all the images of the sequence, however there will be inaccuracies at determining the crystal centre position at oblique viewing angles. This factor is considered in the plots below by presenting results at different tolerances to number of images with the sequence correctly determined to be within the crystal centre. 

Database Results

The 'within crystal' result are cases in which XREC predicted crystal centre is within annotated crystal boundary. In the pie chart shown below the within crystal score is given for the case in which 90% of the images within the sequence are within the crystal boundaries. 
xrec piechart
If we break down the results further we can see that 46% of the test database experiments had all 100% of the image sequence (#=100%) with the crystal centre within the annotated crystal boundary and 16% of the test database were cases in which the entire image sequence did not have any predicted crystal centres within the annotated crystal boundary (#=0%).

XREC piechart broken down into number of images hit
This result is also given in the form of a bar chart and shows how the within crystal score varies with the number of images in sequence.

XREC centering performance broken down into number of images hit

The following pie chart uses the same pie chart template shown above however this time using data from XREC crystal centre position within the annotated loop boundary.
Pie chart showing loop centering results


Whilest the images were being manually annotated a subjective crystal centering difficulty score was given to each image. The plot below shows that XREC performs best on the images judged to be of medium difficulty and less well on images being judged difficult (8) or easy (2) to center. However the fact that difficulty does not appear to correlation with performance may also indicate that the it is not possible to subjectively assess how well XREC is likely perform on a given image.

Dependence of XREC centering performance on subjective difficulty criteria

The plot below shows XREC centering performance in terms of distance from crystal centre. This distance is given as the xrec calculated disance from the annotated crystal centre and is normalised by dividing by the annoated crystal centre to crystal boundary distance. Hence a distance ratio of 0 correspond to a XREC calculated crystal centre position on the crystal boundary and a distance ratio of 1 being the case in which the XREC and annotated crystal centre are the same and therefore the frequency of occurance within the range of 0 and 1 correspond to XREC crystal centres positions within the annotated crystal boundaries.
 
Distance of XREC centre position from crystal centre (expressed in terms of the centre position within the crystal boundary)
The following plot shows the Euclian 3D XREC crystal centre position distance from the annotated crystal centre position in microns.
3D distance in microns between XREC dermined crystal centre and annotated crystal centre
The following 2 plots are based on percentage beam/crystal overlap ratio. These results indicate the expected sucess rate in terms of beam intersection for different beamsizes and crystal sizes. This ratio is based on the beam overlap which is given as the intersection area between a centered (square) beam area and the annotated crystal polygon. The overlap ratio is given by the beam overlap as based on XREC crystal centre position divided by the beam overlap centred on the annotation crystal centre position. These results are given for different beamsizes and crystal sizes and for beam overlap areas > 80% with 80% overlap being considered a sucessfull centering for a given beam size.
The dependance of beam/crystal overlap On crystal size
The following plots shows the beam intersection distribution for different beamsize.
The dependance of beam/crystal overlap on the size of the beam