Evaluation of the ACR MRI phantom for quality assurance tests of 1 . 5 T MRI scanners in Estonian hospitals

Since its discovery, magnetic resonance imaging (MRI) has been one of the main methods of imaging in radiology. So far, there are not many national or international guidelines for MRI quality assurance compared to imaging methods that are using ionizing radiation. American College of Radiology (ACR) has an accreditation program that includes a standardized image quality measurement protocol and a phantom. Seven important assessments of MRI image quality are included with acceptance criteria. The aim of this study was to determine whether using the ACR MRI phantom would be a suitable quality assurance method for MRI systems in Estonia. In order to determine if the ACR MRI phantom would be a suitable quality assurance method for MRI systems in Estonia, six MRI scanners were tested using the ACR MRI accreditation program phantom and method. Information about the image quality was obtained and 33% of the MRI scanners passed all seven assessments. In conclusion, the ACR MRI phantom is a suitable test body for some of the MRI scanners in Estonia and should be considered to be used for regular quality testing. Due to the fact that in some cases the ACR MRI phantom did not fit into the head coil, using the ACR MRI phantom cannot be nationally required and other quality assurance phantoms should also be assessed. Adding signal-to-noise ratio assessment to seven assessment criteria, provided by ACR quality assurance manual, and developing an automated ACR quality assurance procedure would make quality assurance with ACR MRI phantom more beneficial.


INTRODUCTION
* Since the inception of magnetic resonance imaging its influence to healthcare and medical imaging has been enormous.In 2007 there were already more than 20 000 magnetic resonance imaging (MRI) scanners regularly in use [1].In Estonia, 13 MRI scanners were in use in 2011 and more than 61 000 MRI studies were made in that year [2].
MRI is a combination of advanced science and engineering, including the use of superconductivity, cryogenics, quantum physics, digital and computer technology.In short, MRI is a very sensitive imaging method that is based on the amount and properties of water, which is the constituent part in 70% to 90% of most tissues [3].Due to the fact that MRI scanners are greatly complex in nature and have very high accuracy demands, the scanners are very vulnerable to technical and image quality problems.Images from different scanners should fulfil the same technical and clinical standards and that makes regular quality assurance (QA) essential to MRI scanners [4].
So far there are not many national or international guidelines for MRI quality assurance, but substantial work has been done.Special phantoms for QA have been made.Most known are the Eurospin phantoms, MagNET phantoms, and American College of Radiology (ACR) phantoms.Manufacturers of the MRI scanners also make their own quality assurance tests, but protocols, standards and phantoms for these tests are usually model specific and concentrate on the stability of mechanical components and electronic circuits.QA test procedures as well as construction of test phantoms from different MRI systems manufacturers are different and therefore the tests results are not comparable.In contrast, Eurospin, MagNET, and ACR phantoms concentrate on the image quality and are independent of manufacturers and models, making it possible to perform the same quality assurance tests to different MRI scanners.
Developing a nation-wide manufacturer independent QA protocol gives an opportunity to compare different scanners and assure that images from different scanners are with the same quality.Most importantly, the minimum image quality level should be equal in all MRI scanners in the nation.Also test procedures and phantoms of MRI systems manufacturers are not proper to evaluate the essential image quality parameters determined in the national standard EVS-EN 62464-1:2007.
To this day, regular manufacturer independent quality assurance tests for MRI scanners are not made in Estonian hospitals.To establish a nation-wide MRI QA protocol, a suitable MRI QA phantom should be chosen and verified.The phantom should be in conformity with the EVS-EN 62464-1:2007 standard and usable with all the MRI scanners in Estonia.In this research note, quality tests with ACR MRI phantom were made to establish if the ACR MRI phantom is a suitable quality assurance test body and should it be considered to be used for regular manufacturer independent quality testing in Estonian hospitals.

ACR MRI phantom
American College of Radiology has created many phantoms for accreditation purposes.For MRI scanners, two standardized phantoms, large and small, have been created to estimate the technical quality of images [5].Large phantom is designed to be used in head coils and is used for testing most of the scanners.The small phantom is scanned in the knee coil and is only used for extremity-only units.Although the ACR MR phantom was designed for ACR accreditation purposes, different MR sites are utilizing the phantom for MR image quality assurance and system performance testing [6].
Seven important parameters for the assessments of MRI image quality are included with recommended acceptance criteria.These parameters are: (i) geometric accuracy, (ii) high-contrast spatial resolution, (iii) slice thickness accuracy, (iv) slice position accuracy, (v) image intensity uniformity, (vi) percent signal ghosting, and (vii) low-contrast object detectability.All these parameters are specified in EVS-EN 62464-1:2007 standard for routine QA in MRI sites.
The ACR MRI phantom is a short hollow cylinder of acrylic plastic filled with a solution of nickel chloride and sodium chloride (10 mM NiCl 2 and 75 mM NaCl).The inside length of the phantom is 148 mm and diameter 190 mm.Inside the phantom are several structures that are necessary to test the scanner performance [7].
All QA tests were performed once on each scanner during November, 2013.The phantom was laid to the centre of the head coil as a head would be in the coil.For the positioning marks "nose" and "chin" were used.After the first positioning, a non-metallic bubble level was used to make the positioning more precise.For the quality assurance, three different scans were made with parameters, specified by the ACR accreditation program.These scan protocols and parameters are shown in Table 1.Examples of the locator slice and eleven ACR Axial T1 slices are presented in Fig. 1, where after the locator eleven slices follow, starting with slice 1 and ending with slice 11.In this paper the evaluation of images were performed as instructed by the ACR.The evaluation was done with the K-Pacs v 1.6.0DICOM viewer, because it is one of the non-commercial suitable DICOM image viewer software, recommended by the ACR and is used in several imaging departments in Estonia [8].
In comparison with other MRI QA tests, ACR MRI QA tests have shorter image acquisition times [9].Scanning the fixed phantom with previously adjusted protocol takes less than 13 min.ACR MRI QA method is globally known and used.At the moment, there are already two large phantoms and one small phantom in Estonia.

MRI scanners
Six whole body MRI scanners in clinical use in five different hospitals were tested using the large ACR MRI phantom during November, 2013.Three scanners were manufactured by General Electric Healthcare (Optima MR450w, Signa Hde, Signa HDx), two scanners by Siemens Healthcare (Avanto, Symphony), and one by Philips Healthcare (Ingenia).The oldest scanner was from the year 2001 and the newest from the year 2013.All tested scanners had the field strength of 1.5 T.

RESULTS
Out of six MRI scanners two passed all the ACR MRI quality assurance tests.Four scanners failed to reach the acceptance criteria given by the ACR accreditation protocol.Three of them did not pass the image intensity uniformity test and one scanner did not pass the highcontrast spatial resolution test.

Geometric accuracy
With geometric accuracy test it is possible to assesses the accuracy with which the image represents lengths in the imaged subject.For this test, seven specific length measurements were made and compared to the known values for those lengths.These seven measurements are end-to-end length of the phantom that was measured from locator image, diameters of the phantom in vertical and horizontal direction, measured from the first slice and diameters of phantom in vertical, horizontal, and diagonal direction, measured from the fifth slice of the ACR T1 series.Recommended criterion is that all measured lengths should be within ± 2 mm of their true values [7].Measurements, made from the locator, first and fifth ACR T1 slices, are seen in Fig. 1.Measured lengths are represented in Table 2.All scanners passed the geometric accuracy test.

High-contrast spatial resolution
With high-contrast spatial resolution test the scanner's ability to resolve small objects was assessed.For this test, one visually assesses the distinguishing ability of individual small bright spots on the image that actually are three pairs of hole arrays.A pair consists of an upper left (UL) hole array and a lower right (LR) hole array.The hole diameter differs between the three hole arrays.The hole diameter for the first array pair is 1.1 mm, for the second one 1.0 mm, and for the third one 0.9 mm.Recommended criterion is that one can in visual evaluation differentiate holes with the diameter of 1.0 mm at least [7].
The three hole array pairs can be seen on the first ACR T1 slice in Fig. 1, where they are located at the bottom side of the slice.In the images the hole array pair seems like two squares linked together through one corner and filled with small bright dots.The results are represented in Table 3, where the values indicate the size of smallest holes that were resolved.In visual evaluation, scanner 4 did not pass the high-contrast spatial resolution test.

Slice thickness accuracy
With slice thickness accuracy test, the accuracy with which a slice of specific thickness is achieved, was assessed.The prescribed slice thickness, 5.0 mm, was 0.9 0.9 1 1 3 0.9 0.9 0.9 0.9 Fig. 2. Measured slice thicknesses in the slice thickness accuracy test.
compared to the measured slice thickness and the measured slice thickness should be 5.0 ± 0.7 mm.The test was performed on ACR T1 scans and ACR T2 scans [7].
For measuring slice thickness, the length of two special signal ramps were measured.The ramps can be seen on the first ACR AT1 slice in Fig. 1, where they appear as one thick horizontal line in the middle of the phantom.The values for measured slice thicknesses are represented in Fig. 2, where the black lines indicate the acceptance criteria values.All the scanners passed the slice thickness accuracy test.

Slice position accuracy
With slice position accuracy test it was assessed how much do the actual locations of acquired slices differ from their prescribed locations.For this assessment, length difference of special bars in two slices was measured.The bar length differences were measured on the first and eleventh slice from ACR T1 (Fig. 1) and ACR T2 scans where the bars are located at the top of the phantom.The absolute bar length difference should be 5 mm or less, but it is advisable to keep the bar length difference to 4 mm or less.This test requires no analysis of the measurements.The action criteria are specified in terms of limits on the bar length difference measurements.However, because the crossed wedges have a 45° slope, the bar length difference is twice the actual slice displacement error [7].
The results of the slice position accuracy test are represented in Figs 3 and 4. Black lines on the figures represent the maximum difference of bar lengths that is allowed.When the left bar is longer, the slice is displaced inferiorly with respect to the vertex, in that case a minus sign is assigned to the length.All the scanners passed the slice position accuracy test.

Image intensity uniformity
With image intensity uniformity test the uniformity of the image intensity over a large water-only region of the phantom was assessed.For this assessment a value, called percent integral uniformity (PIU), is calculated from the seventh slice of ACR T1 (Fig. 1) and ACR T2 scans.To pass the test, PIU should be greater than or equal to 87.5% for the MRI systems with field strengths less than 3 T [7].
Percent integral uniformity values are represented in Fig. 5.The black line indicates the PIU value that has to be passed in order to pass the image intensity uniformity test.As it is seen from the figure, three scanners (1,5,6) had PIU values less than 87.5% and did not pass the test.

Percent-signal ghosting
With percent-signal ghosting test the level of ghosting in the images was assessed.Ghosting is a type of artefact in which case a faint copy of the imaged object appears on the image and is displaced from its true location.For this test, ghosting ratio is calculated from the seventh slice of ACR T1 scans (Fig. 1) using one big and four small regions of interest.To pass the percent signal ghosting test ghosting ratio should be less than or equal to 0.025 [7].
The values of calculated ghosting ratio are represented in Table 4.All the scanners passed the percent signal ghosting test.

Low-contrast object detectability test
With low-contrast object detectability test the extent of which low-contrast objects are discernible in the images was assessed.For this purpose, the phantom had four slices with a set of low-contrast rows of small disks that differ in size and contrast.Disks can be seen in Fig. 1 on the 8th to the 11th ACR T1 scans.There are recommended criteria for low contrast object detectability test in ACR QC manual.In order to pass the test, it has to be possible to differentiate at least 9 rows of disks on the slices in total [7].
The discs can be seen in Fig. 1 on slices 8-11.The results are represented in Table 5.All the scanners passed the test.

DISCUSSION
Using ACR MRI phantom for quality assurance purpose has many advantages.With ACR MRI phantom it is possible to test many image quality parameters that manufacturers do not test on their regular check-ups.
The ACR MRI phantom is independent of manufacturers, providing the possibility to be used in different MRI scanners and to later compare those test results to one another.In addition, the ACR has an internationally recognized status as a developer of guidelines and standards for radiology devices and QA procedures.The image acquisition procedure is fast and with one phantom it is possible to assess seven different important quality parameters.
Similar studies with the ACR MRI phantom have been made earlier, for example [4,9].In [4], two tests were made and eleven MRI systems were tested.On the first measurements, 91% and on the second measurements 73% of systems passed the tests.In [9], half of the four scanners tested passed the QA tests at the first time and after the adjustments had been made by service engineers, all the scanners passed the QA tests.In [4] it was stated that the image acquisition procedure of the ACR tests was fast and practical and the method proved feasible for quality assurance in the multi-unit imaging centre, where the tests were made.In [9] it was stated that the proposed ACR MRI QA protocol provides a simple and comprehensive assessment of the performance of a MRI scanner.
The authors of the current study, after performing the tests themselves, agree with authors of the studies mentioned earlier.Even though the sets of tests were solid, fast and easy to perform, there were also some shortcomings that emerged.Due to the dimensions and cylindrical shape of the phantom, the test-body did not fit into the head coil models used in medical practice with all three GE systems and the tests had to be made with test coils, manufacturers use for their tests.The same problem was brought up in [4].The fact that the ACR MRI phantom did not fit inside some head coils is considered to be problematic, because using different coils can influence some quality assurance tests of the MRI scanner, for example the image intensity uniformity test.One possible solution would be to consider using the ACR MRI small phantom what is meant for the extremity units and is therefore smaller.This option should be beforehand thoroughly assessed.
The parameters, given for the scans, differ substantially from the parameters used in medical practice and some important parameters that can significantly change the image quality were not stated, for example, the bandwidth.The ACR protocols also did not specify any scan options such as autoshim or image filtering that can likewise have an effect on the image quality.In the ACR MRI phantom test scan protocol all the filters and shim autotuning parameters were switched off.This may be one reason why several scanners under test did not pass the percent integral uniformity test and one did not pass the high contrast resolution test.
In this paper, six MRI scanners were tested with the parameters stated by the ACR MRI accreditation program.Of the tested scanners, 33% passed all the criteria stated by the ACR protocol.Half of the scanners had percent integral uniformity value in the image intensity uniformity test lower than the acceptance value given by the ACR.One possible reason why these scanners did not pass the test is that the phantom was not positioned in the centre of the head coil but instead laid in the bottom of the coil.For future tests, special structures should be designed for every used head coil to position the phantom accurately to the isocenter of the coil.For the scanners that did not pass the recommended criterion, hardware performance tests should be performed, phantom positioning and acquisition parameters should be revised, and ACR quality test should be repeated.When the problem remains, different actions should be performed depending on what type of test was not passed.For example, additional manufacturer tests and calibrations should be performed.
Some of the tests depend on the visual evaluation, in which case the result may differ depending on the evaluator or monitor that is being used.In order to make the evaluation more objective, an automated ACR quality assurance procedure should be developed.Signal-to-noise ratio assessment should also be added to the seven assessments, because of its fundamental position in the MRI quality assurance.Signal-to-noise ratio is also one of the main parameters that are needed to be assessed according to the EVS-EN 62464-1:2007 standard.

CONCLUSIONS
Using the ACR MRI phantom for quality assurance purpose is a feasible way to detect if the tested MRI scanners fulfil the standards of a well-functioning MRI scanner.Even though manufacture specific service programs are used, it is important to use standard phantoms in MRI quality assurance to ensure that minimum image quality level is equal to all MRI scanners in practice.
For better quality assurance, signal-to-noise ratio assessment should be added to the seven assessments.Special structure for each tested head coil should be designed in order to assure that the phantom is positioned in the centre of the head coil.To make the image analysis more objective, an automated ACR quality assurance procedure should be developed.
Despite the fact that the ACR MRI phantom has all the structures to make the QA tests, requested in the EVS-EN 62464-1:2007 standard, the phantom did not fit inside some head coil models.According to the standard, phantom used for QA must fit into the head coil.Due to the fact that in this study the ACR MRI phantom did not fit inside some head coil models, it cannot be made nationally requested and other available QA phantoms should also be assessed.

Table 2 .
Length measurements for the geometric accuracy test

Table 3 .
Results of visual evaluation in the high-contrast spatial resolution test

Table 4 .
Ghosting ratio values in percent signal ghosting test

Table 5 .
Detected rows in the low-contrast object detectability test