This project is submitted for

Description

Diabetic Retinopathy is a complication of diabetes causing damage to the retina, eventually leading to blindness. The cost of state of the art retinal imaging devices required for identifying this disorder lies in the range $10,000 - $25,000. This makes them inaccessible for the population in rural areas or developing countries. We aim to develop a device under $400 which can provide reasonable quality retinal images to clinicians.

Details

Overview

Currently there are over 422 million people worldwide suffering from diabetes. 28.5% of them suffer from Diabetic Retinopathy(1) . 50% of diabetics are unaware about the risk of losing their vision(2). The number of cases of diabetic retinopathy increased from 4 million in 2000 to 7.69 million in 2010 in US alone. Early detection and Treatment can help prevent loss of vision in most cases.

Detection of Diabetic Retinopathy, requires expensive devices for Retinal Imaging , even the cheapest of them costing more than $9000 each. This makes good quality eyecare, expensive and inaccessible to the less privileged. The key idea in the development of OIO(code-named Project OWL) is to provide an affordable solution to help identify DR and hence prevent cases of "avoidable blindness".

What is OIO?

OIO(OWL) is an idea conceived in the Fall 2015 MIT Media Lab Engineering Health Course. Development of the electronics, enclosure, and optics have continued since Spring 2016 at the Srujana Innovation Centre at the LV Prasad Eye Institute, Hyderabad, India. It aims at capturing good quality retinal images using an affordable device, mainly to help make identification of Diabetic Retinopathy accesible to all.

Key Features:

Ultra Low Cost: Under $400, compared to its contemporaries

Open Source and Design: Expands the scope of the device worldwide

High Portability: Weighs less than a laptop

Intuitive interface: No specialized training needed to operate the device, making it perfect for use in rural areas

How it works?

Ophthalmoscopy is a technique by which optometrist analyze and view the retina and its features by either directly looking or imaging, the retinal features through the pupil. OIO captures the images of the retina using the same technique, through a camera.

The OIO uses a 20D lens, mirrors, light source and camera, a raspberry pi and a touch interface to achieve this. The lens and mirror system are used to compress the optical path and focus the light from retina onto the camera. The camera and pi act as the processing unit and display live images on the display allowing the clinician to easily image the retina.

Project Logs

The design models have been 3D printed and assembled. We made two devices, only the outer casings are different in the two models. One is printed on a SLS 3d printer (Nylon PA-12) and another one with an FDM 3D printer (PLA Black).

Both the models are working fine and were able to capture decent retinal images.

Assembled Devices

Earlier version of the device was reconsidered in terms of following aspects:

• Accommodating for different refractive errors

• Aesthetics

• Ergonomics

• Weight distribution

OIO

ASSEMBLY

EXPLODED ASSEMBLY

1. ACCOMMODATING FOR DIFFERENT REFRACTIVE ERRORS

For different people with varying refractive errors, focal length for the 20D lens changes.

For an eye with normal refractive error, the distance between the 20 D lens and the eye has to be 50mm for a focused image of the fundus.

For people with negative errors, the distance goes on increasing and for people with positive refractive errors, the distance goes on increasing.

BEFORE

A. In the earlier iteration,the optic box was fixed at particular position such that the distance between the eye and the 20D lens was 50mm.

B. Position of the device with respect to the patient's eye with a negative refractive error (power) . As the person has some negative error, the distance between the eye and the 20D lens is more than 50mm. Hence the device has to be held away from the face to increase the distance.

AFTER

The optic box moves back and forth inside the device by means of sliders which are integrated as a part of the eye piece and the middle housing.

The box moves by the means of a rack and pinion assembly.

The distance between the eye and the lens can be adjusted by rotating the knob as in the image above.

2. AESTHETICS

The meaning that the appearance of a product communicates helps consumers to assess the product on functional, aesthetic, symbolic or ergonomic motives in this case a medical setting.

A. COLOR and MATERIAL

The subtle colors used, grey and beige, are suitable for a medical device. These colors have a calming effect on the patients.

The material used is medical grade and is safe for use.

The part of the eye piece which comes in contact with human skin has been made out of silicone. Thus it can be cleaned with alcohol swabs after every testing.

B. FORM FEATURES

The physical appearance or form makes the device look safe and easy to use.The basic aesthetics including contours and curves tend to make the product intuitive for the end user.

There are suggestive grips provided at strategic places on both sides to ensure correct handling and positioning.

These provide a clear boundary between the interaction of the patient with the device and the clinician with the device.

The heat sinks provided on the outer housing near the R-pi add to the aesthetics of the device.

3. ERGONOMICS

A.Symmetry along horizontal axis

The device has to be symmetric along the horizontal axis.

This is because only one eye is to be tested at a particular instance.

The cavity near the other eye is closed . To test the other eye, the device has to be flipped.

B.PLACEMENT OF SWITCHES AND KNOBS

The click switch is placed ergonomically at the side of the device.Thus, image can be captured for both the eyes by flipping the device.

The knobs for moving the optic box are placed at center bottom and center top of the device with a rod connecting the two

C. ERGONOMIC EYE PIECE

The eye piece plays the most important role in the device.

In a conventional fundus camera, the face of the patient is held steady by means of a chin support and the camera lens is placed perpendicular to the face setting.

As OWL is a hand held device ,holding the device perpendicular to the face plane was quite a task. If the device is not held perpendicular, only partial fundus can be observed.

Hence, the eyepiece had to be designed in such a way that it is a 95 percentile ergonomic fit and aligns perfectly onto the face perpendicularly.

Several mock prototypes were tried and tested to get this particular ergonomic shape.

Also the protruding end at the temples ensure a further assurance of a perfect fit.

4. WEIGHT DISTRIBUTION

The earlier prototype had the battery bank at the front of the device.

Hence, the device tended to sag(hence it was not perpendicular to the face plane)

The messy wires inside the device can be taken care by decreasing the lengths of the wires. All the connectors should be standardized so they can be easily reproducible.

We can start with the PCB. The earlier PCB was used for prototyping. It has 8 slots for the multiple LED's for testing to get the proper illumination on the Retina. Now we know, we only need single LED to shine the Retina. So the extra slots can be removed and the PCB can be compressed.

LED Placement

If you see the LED, it is placed on the top of the camera by which we are capturing the image of the left eye's Retina. If we need to get the same standard image on the right eye as well, then we need to place a LED on the bottom so that when the device flipped it comes on the top of the camera. Thus we can get standard images from both eyes.

TWO LED's

So a new PCB with only 2 LED ports can be made.

PCB Design

PCB Assembly

Touch cable on R-Pi

The cable for touch from one of the R-Pi USB ports to the screen can be by passed. By finding the corresponding solder connections for a USB port on the R-Pi we can solder the USB cable's wires directly to those points and decrease the length of the wire.

Power Input for R-Pi

The micro USB power input to the R-Pi can be by passed by soldering two terminals to the +/- on the R-Pi and connect them on the PCB

Power Cables from Power Bank

Power bank has 2 USB female ports. One is 2.0A and another 1.0A. The PCB need to drive the R-Pi and the LED's so 2.0A can be directed to PCB. Instead of desoldering the USB ports on the power bank PCB connect a 2 pin female connector to a USB cable (Soldering micro USB female to the PCB is time taking). This will connect the power bank PCB through USB and to the main PCB it acts as a power input terminal with a 2 pin connector.

LCD screen couldn't take sufficient power from the touch port. As a result the touch malfunctions once R-Pi starts accessing the camera. So we need another power source so that's why we are using the 1.0A for the Screen.

R-Pi and screen works on the same power source which is always on. For the R-Pi a switch slot is already given. Now the screen works on an independent power source which should be connected to a switch. So we need to add another switch to the screen power cord (Can add a 5V relay but it drains the power bank). Slit the cable and add a slide switch any one of the terminals.

The end of the cable is micro USB which is covered by a thick plastic layer. Because of this the cable can't bend and fit into the enclosure of front piece. So chip off the plastic with a knife and insulate it with a heat shrink.

Here we'll mention about the two major updates that were done which may not be visible normally.

1) Changing of the pins : The GPIO pins have been changed for the LEDs . This is because the printed PCB was fitting into the GPIO only in that manner. Although there was no major update in the code that was there but the pins are now changed and it can be something hard to find.

2) Allowing multi-threading in the flask app : The flask app was crashing every now and then whenever we wanted to get the theia grading. This was later solved using the following attribute while running the app.

app.run(host='0.0.0.0', threaded = True)

Earlier it was giving an 'Internal Server Error' if Theia returned no value or timed out.

For making the device screen people with different refractive error the optic box had to be moved inside the casing. To achieve this, a sliding system was created by making positive rails on the optic box and negative rails inside the casing.

Optic Box

The box has stress at the edges as its weight is concentrated at the rails. Hence the strength of the edges of box had to be increased. Holes at the edges were provided for mounting screw fasteners to increase the strength of the optic box.

Sliding System with Rails

Rack and Pinion Mechanism:

To impart motion to optic box, rack and pinion mechanism has
been used. The rack has been placed on the optic box and the pinions are
assembled inside the eye piece with a rod which transmits motion to different
planes and eventually to the racks. The pinions are assembled inside the eyepiece with the help of a bracket. The bracket holds the pinion in its place while giving it freedom to rotate in the plane.

Rack and Pinion Assembly

LCD:

The Waveshare 5 inch LCD has mounts to assemble the LCD onto a surface using screw fasteners.

LCD with mounts

But, to accommodate the LCD inside the front casing we have to cut the mounts that come with the LCD.

LCD without the mounts

To assemble the LCD with the Front casing we made a support for the LCD which would act like a mount for the LCD.

LCD with printed support

LCD Assembled inside the middle casing

PCB of Batteries:

The PCB of the Batteries has to be placed inside the Middle casing. To assemble it inside the middle casing a mount has been created on which the PCB is fastened with the help of nut and bolt fasteners.

Batteries with the PCB

PCB with the mount

Click Switch:

The Click switch has been mounted inside the middle casing using a mount. It is first mounted into the mount and is then assembled inside the middle casing using screws.

Mount for Click Switch

Click Switch

Click Switch with mount

Click Switch Assembled inside the middle casing

Knob:

The knob made earlier was slippery, thus the total motion imparted was not transmitted due to slipping. The knob is given texture to increase grip and friction and help in transmitting motion to the rod.

Textured Knob

Toggle Switch:

To regualte power to raspberry pi and LED's, a switch has been incorporated in the pcb. But there wasn't any switch to on/off the LCD screen. Thus a double switch was made which would facilitate as a switch to both the screen and raspberry pi,LED.Here are some of the images showing the making of a double switch.

The Cable with a switch

Double Switch

Double Switch inside the Middle casing

Camera Spacer:

To mount the camera to the Optic box a 4mm mount is used to create space between the optic box and the camera. This is needed to ensure that the camera is perfectly centered and parallel to the plane of sight.

This log talks on how to assemble the internal circuits of Open Indirect Ophthalmoscope.

Dismantling:

Crack open the power bank (13,000 mAh power bank with power output of DC 5V, a device with two output USB ports such that one gives a current of 2.1 Amps and the other 1 Amp.)

The reason for choosing a power bank with two such outputs is that the LED circuit and the RasPi together need a higher current, which in this case is being powered by the 2.1 Amp port and the display is powered by the 1 Amp port.While doing this, care should be taken that the circuit is not damaged. Power cables are soldered onto the PCB, therefore after cracking, do not jerk open the bank.Here are some references for the power banks that can be used.http://amzn.to/2cNw0avhttp://amzn.to/2d4wjNt It is preferable to use a power bank with the USB A Type and Power Bank Charger - USB B Type on adjacent sides. This makes the assembly easier.

Remove the batteries carefully from the casing. There should be five batteries, remove one battery out from the circuit by cutting the metal plate interconnecting the batteries.

Once this has been done, separate out the batteries into sets of two with both sets electrically parallel to each other. At this point connect two sets of wires connecting two sets of batteries, while doing so place the heat shrink material in the wire, this will be used to make the wires appear seamless and give them strength once all the connections have been soldered.

The separation of batteries is done so as to provide an even weight distribution throughout the chassis. Placing two sets of two batteries each makes the weight balance out horizontally.

These wires should be placed such that the ones connecting the positive ends are slightly longer than the negative ones. Carefully pass the positive wire in the crevice between the batteries. Solder the negative terminals. The resultant battery setup should look like this:

This allows the batteries and the wires to seamlessly fit in the battery encapsulation groove in the 3D printed assembly. Carefully place the batteries and check the wire lengths.

Now take the PCB from the power bank and increase the length of the wires connecting the batteries to the PCB. They should be connected onto a terminal 'B+' for the positive terminal and 'B-' for the negative terminal respectively. Solder the terminals and seal the solder with silicone glue to give the entire joint some strength.

On the same PCB extract the USB ports by heating their soldered ends (4 Connections) from one side using a hot air blow gun or a solder gun and pulling from the other end simultaneously. Be careful to not pull too hard, as this might cause the circuit to get damaged.

After the USBs have been extracted, it's time to solder the cables connecting the power supplies to the PCB and the LCD.

Take a micro USB cable and cut off the non micro USB end. Strip this end and peel the wires out. There should be two wires (if you are using a charging cable) or four wires (if you are using a charging & data transfer cable). The standard colour combination is Red - for the positive connection and Black - for the negative connection. Solder the red wire to the positive end and black to the negative/ground on the power bank PCB at the 1 Ampere port. This cable will supply power to the custom LCD.

Take another set of two wires (red & black preferably) and solder the red end to positive and black end to negative/ground of the Power Bank PCB's 2.1 Ampere port. Take the other end of the wire and attach two female headers on to it. These will be connected to the custom PCB's power supply. This will be used to supply the power to the LEDs illuminating the retina and to the Raspberry Pi.

Soldering the Ras-Pi Shield PCB

Place the 2N222 Bipolar Junction Transistors (BJTs) carefully on the PCB. Most PCBs come with a hemispherical marking and a flat end marking to ensure correct orientation. Though this may not be true in all cases, in such a situation one way to understand...

Pain points in the present model and ideation for next prototype:

The earlier device housing was made in three part assembly which made it easy to assemble and disassemble during bugging. So the same method will be used in the coming prototype as well. The three parts are

Eye piece (Through which the camera see's the eye)

Middle part (This houses the optical setup with the camera)

Front part (This contains screen, Rpi and PCB's)

Detailed description of pain points and ideation process.

The user interaction of the device:

Eyepiece fitment on the examinee eye

The present model doesn't have proper grip to hold it. This caused
slipping of the prototype while testing . Proper gripping has to be given to
hold the device while testing.. This can be solved by giving embosses with
ergonomic hand detailing on the middle part of the device.

Image capturing switch is too far from the screen and is not easily accessible
while taking the picture. The device front part is bulkier than the palm so
reaching for the touch screen is a bit difficult with simultaneously focusing the retina and taking the image.

Heat buildup due to processors running on camera and Rpi. So vents have to be provided on the outer surface for convective heat transfer through air.

The focusing issue:

Same problem as in VR headgear, because of different refractive
errors the lens should be moved front and back to get the retina in camera's
focus. In our case the lens alone shouldn't be moved as it changes the field of view
of retina. The whole optics box has to be moved to compensate for the
refractive error. This can be achieved by adding a sliding mechanism to the box
and a rack and pinion mechanism to control the distance between the eye and the
lens.

Device orthogonality with the examinee:

While capturing the image the objective lens(20D) plane has to be
parallel to the subject's eye lens plane. In the present prototype this is not
happening because of uneven mass distribution towards the back.If not parallel we get distorted images of the retina as the camera and light won't focus the retina properly. By keeping the device orthogonal to the examinee we can ensure the planes of 20D lens and lens of the eye are parallel to each other.It can be done by two methods one by adding a headband as normally used in VR headgears. Other option would be arranging components in such a way it redistributes the whole weight and achieve orthogonality as keeping an head band with the device needs sterilization every time after use and feels little invasive for the patient.

Selecting proper prototyping technique:

Present prototype was made on an FDM 3D printer. After printing on an FDM printerwe
need to remove supports and excess plastic either with cutting tools or
grinding tools. Lot of postprocessing has to be done in getting smoother
surface. This causes uneven dimensional tolerances. Once we add the sliding mechanism
and three piece assembly device the surface should be really smooth for movements and ease of assembly. This can be overcome by printing those parts
with an SLS printer.

Till now we had been using an older OS incompatible with RPi 3 since the display could have only worked with that image. Additionally it caused the trouble where the onboard or external wifi dongles were not able to work with it.

Since Waveshare introduced the new RPi 3 compatible image, we tried to build the entire system ( dependencies ) again onto the new OS.

6) Configured crontab to start the required processes at system startup

7) Installed Chromium

After performing all these we were able to make a " Burn and Run " image of the device. Now this image can be burnt onto any sdcard and inserted into the pi to be used for making the OWL device and we are ready to go.

The machine learning system on cloud was working well on the website theia.media.mit.edu - our goal is to integrate this into the device via API, so that whenever someone takes images they can choose to grade it by sending it on cloud. The natural limitation of this is that the user needs access to the internet, which may not be available in remote areas, and so we would eventually have to move the processing of images onto the raspberry PI itself. At the present moment, for a proof of concept we would use the cloud-based system.

This is written in python as a seperate module, with a function grade_request() that is callable, with the filename supplied as an argument. The API takes a multipart form, in which we send the file object (C-style, returned by open() in python) using the requests module. The API request requires a unique token which is generated by the website - this would help us identify the institution/device making the request.

The server returns a 200 reponse, and a JSON object containing the grade as a float (from 0.0 to 4.0). To test it out, the following table surveys a few test images and their responses from the server:

Image

Grade

2.08

0.09

0.13

0.33

0.08

2.27

3.08

2.43

As can be seen, even the first image, which had a lot of noise (i.e. non-retina artifacts like the lens holders and other parts in our device which had not been computationally removed) gave a result which matched fairly well with what we would expect. The algorithm was robust enough to not be confused by the whitish streaks as well as the black dots (artifacts - usually dust on the camera or lenses) in the 4th image, correctly labelling it within the healthy bracket (less than 1.0).

These tests worked quite well on high-quality images from fundus imaging devices in the clinic. We would like to evaluate the performance of this system on images taken by our device. For this we need to convert our images into a format which is similar to what the clinical standard devices would output - which would start by understanding the limitations in our device's imaging capabilities. The fundamental differences between OWL images and fundus images are:

The glare spots at the center - these would be removed by the computational inpainting method described earlier.

Our device's images are limited to 30 degree field of view (FOV) - which need not be central. This is not a major limitation as this FOV gives enough information for a nonspecific retinal examination.

Our images would have radial distortion due to the 20D lens and the M12 camera lens of the raspberry PI. This would result in slightly different edge distortions compared to a traditional fundus imaging device. To a human examiner, this would make little difference - but to the machine learning algorithm, this may have a non-negligible effect which we would have to investigate.

Point (1) would have the greatest effect on the DR score - since the white spots may be confused by the algorithm to be due to DR. The following table shows the scoring at different stages of processing:

Image

Process performed

Grade

None (raw image)

0.15

Extracted the fundus from the background

0.12

Removed the glare from the image

0.16

Interestingly, there was very little variation between the three images as far as scoring by the algorithm was concerned. However, we would need to conduct a large study on patients and healthy volunteers comparing images from our device and from clinical standard devices to really quantify how closely in agreement the DR grading is. However, for the purposes of a first stage screening device, the automation system broadly classifies images quite well.

At this point, we're getting great images from our device and multiple retina specialists agree that they're able to see the features they need to make a diagnosis. But clinical experts may not always be available to give their opinion for the massive number of patients during a field testing or a mass screening program. We want our device to have automated algorithms built-in which would allow the image to be "graded" by severity of the condition (Diabetic retinopathy).

We implemented a version of some of the winning networks in the Kaggle challenge in Torch, and trained them using rack-based training machine running two Nvidia Titan X GPUs (24 GB total VRAM), 64GB of RAM, and 24 core CPU (similar to those available from Amazon Web Services). Training took about 2 days. After training the network we converted the model to run on both a CPU or GPU based machines. Running the service requires less substantial computing resources, our service currently runs off a desktop machine with 16GB or RAM, 8 cores, and a Nvidia GTX 760 graphics card (2 GB VRAM) in order to comfortably handle 12 image requests/second. For context, if everyone in the world was scanned once a year (7 billion), then we would only need 20 prediction machines, or about $20k (+supporting infrastructure). Finally, to complete the service, we built an API to serve image requests. The service consists of the predictor, written in torch, and a web server, written using the python flask framework. The server receives images, verifies files and sends a pointer to the image data over a socket using TCP to the predictor, which responds with the predicted grade. This configuration allows us to run predictors and the web server on different machines for scale, but currently our service runs off a single desktop. In the future, our CPU based models can run on embedded ARM hardware such as a Raspberry Pi.

In order to interact with the web service, we created a client library in javascript and python. A user makes a profile and is given an API key from our web server. Through this, the processed jpg image is sent as a multipart form. The server processes the image and returns a json object. The grade is returned as a float between 0 and 4, with the following interpretation:

0 - No Diabetic Retinopathy (DR)

1 - Mild

2 - Moderate

3 - Severe

4 - Proliferative DR

The OWL client receives this data and converts it into a simple UI object which is easy to interpret by a field worker or layman.

Can you replace lens and flat mirror with a curved mirror made by gluing a ring onto a mylar sheet (heat blanket) and forming a dome by inflating it slightly - which makes a parabolic mirror. Then pouring a resin(?) support behind the mylar to make the curve permanent when it dries...

The lens is an achromatic doublet so I don't think that would be possible... we simply cannot allow any abberations that would affect image quality. What would the advantage be in doing what you suggested?