This post introduces GoOvid (open-source github link) – a complete redesign of the original Ovid written in Go. I started this as a side project before I stareted my PhD to learn writing Go and gain some practice building prototype systems. It is work in progress, and it does not yet have the full capabilities of Ovid, such as dynamic reconfiguration. However, GoOvid is in a useful state where it can host arbitrary agents and protocols. A README can be found on the github page, so I will avoid rambling about its usage here. You are welcome to take a look, and contact me regarding any questions or comments!
]]>Recently, I had to setup this entire website all over again because how I had first done so was causing issues with upgrading its themes. While migrating to a new repo was a bearable task, what took some fumbling was hooking it back up with Travis CI, the continuous integration tool I use to automatically build and deploy this site. So to save ourselves from crawling over documentation all over again, this post will be a tutorial of how to deploy a Jekyll website to GitHub Pages using Travis CI. If you are not already familiar with Jekyll and GitHub pages, here is some great documentation.
Before we dive into the mechanics, it helps to understand why we want to use Travis CI with Jekyll and GitHub Pages. Indeed, GitHub Pages is capable of building your Jekyll website from its source repo. The catch, however, is that GitHub is wary of executing third-party Ruby plugins. One way around this is to first build your site locally, and GitHub pages will happily deploy the pushed result. But Travis CI helps us avoid this extra step – by hooking it up with our source repo, Travis CI can automatically build it and push it to a deployment repo whenever there is an update. We also won’t have to maintain a “_site/” directory in the source repo. Neat!
For the purpose of this tutorial, I assume that alice/website_source
is the source repository for our website, and we want to deploy to the repo alice/alice.github.io
.
The first step is to create an account on Travis and link it with your GitHub account. You should then see a list of your GitHub repositories in your Travis home page. Simply click on the toggle to activate Travis for alice/website_source
.
Crucially, we now want to give Travis permission to pull and push to your GitHub repos. To do so:
alice/website_source
.MY_WEBSITE
.In the root directory of your source repo, you need to include a file called “.travis.yml”. This instructs Travis what to do at each stage of the job lifecycle. Details can be found here, but for now, this is the file we want to have:
language: ruby
cache: bundler
before_install: gem update bundler
install:
- bundle install
script:
- "./setup"
branches:
only:
- build
sudo: false
This tells Travis the following:
gem update bundler
and then bundle install
, to make sure we are using the latest gems.build
branch of alice/website_source
.I find the last point useful in maintaining an always-working master
branch, by avoiding pushing new changes to my master
branch only to later have Travis tell me that it breaks the build.
Finally, we write the “setup” script that we want Travis CI to run whenever we push a change to “alice/website_source”. This is the script in the ./setup
command we specified in the YAML file. The script and explanation I present here is credited to Paul D’Ambra.
set -e
DEPLOY_REPO="https://${MY_WEBSITE}@github.com/alice/alice.github.io.git"
function main {
clean
get_current_site
build_site
deploy
}
function clean {
echo "cleaning _site folder"
if [ -d "_site" ]; then rm -Rf _site; fi
}
function get_current_site {
echo "getting latest site"
git clone --depth 1 $DEPLOY_REPO _site
}
function build_site {
echo "building site"
bundle exec jekyll build --trace
}
function deploy {
echo "deploying changes"
if [ -z "$TRAVIS_PULL_REQUEST" ]; then
echo "except don't publish site for pull requests"
exit 0
fi
if [ "$TRAVIS_BRANCH" != "build" ]; then
echo "except we should only publish the build branch. stopping here"
exit 0
fi
cd _site
git config --global user.name "Travis CI"
git config --global user.email alice@example.com
git add -A
git status
git commit -m "Lastest site built on successful travis build $TRAVIS_BUILD_NUMBER auto-pushed to github"
git push $DEPLOY_REPO master:master
}
main
First, the script sets the variable DEPLOY_REPO="https://${MY_WEBSITE}@github.com/alice/alice.github.io.git"
, where MY_WEBSITE
is the name of the Environment Variable we set in Travis in step 1.4.
Next, the script runs four functions in sequence:
clean
: delete the “_site/” directory if it exists.get_current_site
: clone the latest revision of the deployment repository into “_site/”.build_site
: run Jekyll build.deploy
:
build
branch.With all the above in place, any push to the build
branch of my source repository will trigger a job in Travis CI. The fancy live log and busy yellow icon show Travis hard at work. When the built site is pushed to my deployment repo, the job will turn a satisfying green.
With some experience, Travis CI is a joy to use. I can see how beyond our current context, Travis and can be used for automated testing and deployment in my future projects.
]]>Looking to dive into distributed systems research, I recently started learning how to write TLA+ specifications. TLA+ is a specification language used to mathematically describe an algorithm, and is adopted in both academia and industry. It provides functionalities to formally verify algorithm correctness. Precisely, it allows us to check if an algorithm satisfies the properties we want it to satisfy.
In this post I shall discuss a TLA+ specification of Consensus I wrote. This specification describes the properties of Consensus at its highest level. That is, we are not concerned with how a specific protocol can achieve Consensus, but what properties should any solution to Consensus possess. It is thus useful for now to forget about mechanisms such as message-passing; a system that solves Consensus using extrasensory perception solves it equivalently as one using TCP.
The Consensus problem is easy to state and understand. Yet it is the basic building-block of computer systems requiring coordination between agents.
Fundamentally, a protocol that solves Consensus in a synchronous system permitting crash failures must have the following properties.
Agreement and Integrity are safety properties: they dictate that inconsistent decisions cannot occur. Validity and Termination are liveness properties: they prevent processes from trivially deciding, say, 0 in all cases, and from not halting.
For our present purpose let us consider, without loss of generality, Binary Consensus, in which each participant proposes either 0 or 1.
To model Consensus mathematically as in TLA+, we adopt a state machine approach. Each participant in the protocol undergoes the following state transitions:
All processes begin in the working state. They then each propose a value, either 0 or 1, to the system. After which, all correct processes then deterministically decides on one of the proposed values (we do not yet care how).
To express these state transitions as math, we begin by defining some constants and variables. In TLA+, we write
CONSTANTS PCS \* The set of all processes
VARIABLES states, \* states of each process in PCS
proposals, \* proposals of each process in PCS
pset, \* set of all proposals
decisions \* decision of each process in PCS
\(PCS\) and \(pset\) are mathematical sets. It may be intuitive at first to think of the variables \(states\), \(proposals\) and \(decisions\) as arrays in a programming language, mapping each process id to its respective state, proposal and decision. However, such intuition is not correct. TLA+ does not operate in the realm of memory addresses. The language of TLA+ is mathematics, and in math, arrays are functions with domain \(\mathbb{N}\) (the natural numbers).
But unlike arrays, TLA+ functions do not restrict us to any particular domain. Here, I shall define \(states\), \(proposals\) and \(decisions\) as functions mapping \(PCS\), the set of all processes, to their respective state, proposal and decision values.
At the heart of all state machines, and indeed any TLA+ spec, are statements describing the initial state of the system, and how it transitions from one state to the next. In my model, while each process is a state machine, the entire system as a whole is itself a state machine, and it is this global state machine we describe in TLA+. Therefore, we can think of the behavior Consensus, and indeed any system, as a sequence of state transitions
\[S_0 \rightarrow S_1 \rightarrow S_2 \rightarrow S_3 \rightarrow ...\]In TLA+, we define \(init\) as a boolean formula describing the properties of the initial state \(S_0\). I describe the initial state of the Consensus as so:
init == /\ states = [p \in PCS |-> "working"]
/\ proposals \in [PCS -> {0, 1}]
/\ pset = {}
/\ decisions = [p \in PCS |-> -1]
This is a logical conjunct of four statements
Note that in each statement, we are not performing variable assignments. Instead, statements 1 through 4 are boolean statements. For instance, statement 1 asserts that \(states\) is indeed the function described. And thus, the statement we call \(init\) is a boolean statement asserting the properties of the initial state of Consensus – any system at its initial state \(S_0\) satisfies the initial spec of Consensus if and only if \(init\) is true when applied to \(S_0\).
While \(init\) asserts what must be true in \(S_0\), the \(next\) formula describes the state transition
\[S_i \rightarrow S_{i+1}\]Thus intuitively, if \(init\) is the base case for an inductive proof, then \(next\) is the inductive step. If \(init\) is true on \(S_0\), and \(next\) is true on \(S_i \rightarrow S_{i+1}\) for all \(i = 0, 1, 2, 3...\), then our system behaves correctly as specified.
For my Consensus spec, I split \(next\) into two possible actions of the system – any process can either choose to propose a value (if it has not yet done so), or decide a value (after all processes have proposed). I write \(propose\) as so:
propose(p) == /\ states[p] = "working"
/\ states' = [states EXCEPT ![p] = "proposed"]
/\ pset' = pset \cup {proposals[p]}
/\ UNCHANGED <<proposals, decisions>>
Again, it is the logical conjunct of four statements that say that in a transition \(S_i \rightarrow S_{i+1}\) where process \(p\) proposes a value:
Likewise, I express \(decide\) as:
decide(p) == /\ ~ \E r \in PCS : states[r] = "working"
/\ states[p] = "proposed"
/\ states' = [states EXCEPT ![p] = "decided"]
/\ decisions[p] = -1
/\ decisions' = [decisions EXCEPT ![p] = CHOOSE x \in pset : TRUE]
/\ UNCHANGED <<proposals, pset>>
It describes the transition where process \(p\) proposes a value, and is the logical conjunct of:
Finally, we can express \(next\) as simply
\[next \equiv \exists p \in PCS : propose(p) \lor decide(p)\]which means that in any step \(S_i \rightarrow S_{i+1}\) there exists a process \(p\) which either proposes or decides a value. When such a process does not exist, then the protocol halts. In TLA+ syntax, that’s
next == \E p \in PCS : propose(p) \/ decide(p)
The protocol is complete. Now we want to make sure that if \(init\) and \(next\) are true on
\[S_0 \rightarrow S_1 \rightarrow S_2 \rightarrow S_3 \rightarrow ...\], then we satisfy the spec of Consensus that is validity, agreement, integrity and termination. TLA+ allows us to do exactly that by expressing these as invariants.
But first, we also want to make sure we did not break any type requirements by specifying the invariant \(typeOK\). I shall omit detailed descriptions as the syntax is similar to what I have described above.
typeOK == /\ states \in [PCS -> {"working", "proposed", "decided"}]
/\ proposals \in [PCS -> {0, 1}]
/\ pset \subseteq {0, 1}
/\ decisions \in [PCS -> {-1, 0, 1}]
Finally, we have the invariants
validity == \E v \in {0, 1} : (\A p \in PCS : proposals[p] = v)
=> \A q \in PCS : (states[q] = "decided"
=> decisions[q] = v)
agreement == \A p1, p2 \in PCS : ~ /\ decisions[p1] = 0
/\ decisions[p2] = 1
integrity == \A p \in PCS :
(states[p] = "decided"
=> \E r \in PCS : proposals[r] = decisions[p])
specOK == /\ validity
/\ agreement
/\ integrity
Here, the only syntax we have yet seen is =>
, which represents logical implication. Therefore, the menacing description for \(validity\) means, quite simply, that, if there exists a value \(v \in \{0, 1\}\) such that everyone proposed \(v\), then at any time for all processes \(q\), if \(q\) has decided, then \(q\) must have decided \(v\).
But what about termination? Well, termination means that the protocol halts. In TLA+, that is characterized as a form of deadlock, and TLA+ has built-in detection for that.
We then set up a TLA+ model as so, telling it to check the behavior described by \(init\) and \(next\) against the invariants, and for deadlock.
Running the model tells us what we have is correct.
It does seem that the Consensus protocol I have described works like magic. There is no message passing, and processes spontaneously decide on a value from a global set. Yet this is the goal. As emphasized above, this is a high-level description of what Consensus should do, and not how it achieves it.
Of course, TLA+ also allows us to specify a particular implementation of Consensus, complete with message passing and the like. But the point of having done as we did here is that using TLA+, we can later check any specific implementation against this high-level specification we have written. In other words, a specific instance of Consensus is correct if it implies that the general spec is satisfied. I will explore doing that in later post.
Lastly, TLA+ lets us pretty-print our spec, so it looks nicely formatted in latex-style math. You can check out the pretty-printed version of this spec, and the code itself, on my GitHub page.
]]>Well, such times will be vanquished. This Christmas, I present to all, the product of my crusade against unproductivity, my final procrastination activity to end all procrastination –
A Python script that blocks websites : https://github.com/TonyZhangND/website-blocker
It blocks any website you tell it to, eliminating time-wasting targets such as Facebook, YouTube, Reddit, etc while you are working. You could also turn it off when you are ready to waste your time again.
The script works by modifying the hosts file in OSX. Understandably, one could achieve the same effect manually with one’s favorite text editor. But hey, I get to procrastinate on studying for the GRE by writing a nifty Python script!
Merry Christmas ~
]]>The problem we are trying to solve is find the exact locations of cells undergoing mitosis in histology images. These are the cells that are annotated with green arrows. In other words, given the Before image above, we want to extract data shown in After.
In Part 1, I have detailed how I used template matching in openCV to obtain bounding boxes on the arrows with 100% accuracy. In this post, I will describe my algorithm used to transform the bounding box information into the coordinates of the arrowheads.
We use the code in Part 1, but with one important change:
import numpy as np
import cv2
import os
from scipy import misc, ndimage
from multiprocessing import Pool
import pickle
import math
import re
import csv
PKL_DIR = {path to pickle files}
LOC_DIR = {path to final csv files}
{import functions from Part 1}
def match_and_pickle(img_name):
""" Apply template matching to img_name in IMG_DIR, using
tmpl{0...359}.png and mask{0...359}.png.
Output: Coordinates of the top left corner of matching templates,
in the format [(deg, [points])], saved in img_name.pkl.
"""
print('Matching img %s' %img_name)
img_rgb = cv2.imread(os.path.join(STRIPPED_DIR, img_name))
img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
matches = []
for deg in range(0, 360, MATCH_RES):
tmpl = cv2.imread(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg),
cv2.IMREAD_GRAYSCALE)
mask = cv2.imread(os.path.join(MASK_DIR, 'mask%d.png' %deg),
cv2.IMREAD_GRAYSCALE)
if tmpl is None or mask is None:
print('Failed to match for tmpl %d.' %deg)
else:
w, h = tmpl.shape[::-1]
res = cv2.matchTemplate(img_gray, tmpl, cv2.TM_SQDIFF, mask=mask)
loc = np.where(res < MATCH_THRESH)
pts = zip(*loc[::-1]) #top left corners of arrow bounding box
if len(pts)>0:
matches.append((deg, pts))
file_name = os.path.join(PKL_DIR,
"".join(img_name.split('_')[:-1])+'.pkl') #eg M21.pkl
with open(file_name,'wb') as f:
pickle.dump(matches, f)
return
Here, I replaced the function match_and_draw()
from Part 1 with match_and_pickle()
. The difference is that instead of drawing the bounding boxes the images, we save the location the arrows and their orientations as pickled data.
Recall that we have 360 arrow templates, one for each degree of orientation, with which we perform template matching. For each image, we save the data in the following Python tuple list format:
[(deg, [(x, y), ...]), ...]
,
where deg
is the orientation of the template that matched, and (x, y)
are the coordinates of the matching bounding box. Note that these are the positions of the top left corners of the bounding boxes.
I can visualize then the bounding boxes with the following code, which loads the pickle data generated by match_and_pickle()
and draws the bounding boxes on a given image.
class Color_Iterator(object):
colors = [(0,0,255), (255,0,0), (204,204,0)]
i = 0
def next(self):
self.i += 1
return self.colors[self.i % len(self.colors)]
CI = Color_Iterator()
def draw_from_pkl(img_name, output_path='./'):
""" Draws bounding boxes on stripped ver of img_name with matches from
img_name.pkl.
Saves output image in output_path.
Example: draw_from_pkl(M21.jpg, './') draws boxes on M21_s.jpg, and
saves output as M21_r.jpg in the current directory.
"""
img_name = ''.join(img_name.split('.')[:-1])+'_s.jpg'
img_rgb = cv2.imread(os.path.join(STRIPPED_DIR, img_name))
pkl_name = os.path.join(PKL_DIR,
"".join(img_name.split('_')[:-1])+'.pkl') #eg M21.pkl
with open(pkl_name,'r') as f:
matches = pickle.load(f)
for (deg, pts) in matches:
tmpl = cv2.imread(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg),
cv2.IMREAD_GRAYSCALE)
w, h = tmpl.shape[::-1]
for pt in pts:
cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), CI.next(), 1)
img_name = "".join(img_name.split('_')[:-1])
cv2.imwrite(os.path.join(output_path, img_name+'_r.jpg'), img_rgb) #eg M21_r.jpg
The above code produces images that appear like so:
Here, I used Color_Iterator
to iterate the color of the boxes drawn. This is to highlight an important feature: There are multiple matches for each arrow. This is because two templates a degree apart may both match on the same arrow; or one template could match many times on the same arrow, each a few pixels apart.
This fact is important later on because it means that there is a many-to-one mapping from matches to arrows. Since we only want one point per arrow in our final output, we need to coalesce a bunch of matches per arrow into one single point.
One important question is still unanswered. Currently, we only have information regarding the location of the top left corner of the bounding boxes. As such, how do we transform these points into the position of the arrowheads? With the orientation of the arrows retrieved from our matching algorithm, this can be done with simple trigonometry!
Consider the above schematic. The dashed box represents the bounding box on the arrow. \(\alpha\), \((x, y)\), \(width\) and \(height\) are known quantities, where \(\alpha\) is the rotation of the matching template, and \(width\), \(height\) are the dimensions of the bounding box.
The center of the bounding box \((cx, cy)\), which is also the center of the arrow, is:
\[(cx, cy) = (x+\frac{width}{2}, y+\frac{height}{2})\]Then, the quantities we are after, \((px, py)\), are:
\[px = cx - l \cos(\alpha)\] \[py = cy + l \sin(\alpha)\]where \(l\) is the practical pixel radius of the arrow. It is longer than the actual half-length of the arrows because we want \((px, py)\) to be slightly ahead of the arrow tip.
Note that in this case, I have defined the negative x-axis to be zero degrees, and the angle increases anti-clockwise.
RADIUS = 42
def transform(matches):
""" Transforms a list of (deg * coordinate list), where the coordinates
are of the top-left corner of the tmpl bounding boxes, to a list containting
the coordinates of the tips of the arrows.
Returns the list of transformed coordinates.
"""
tips = []
for (deg, pts) in matches:
tmpl = cv2.imread(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg), 0)
w, h = tmpl.shape[::-1]
rad = math.radians(deg)
for pt in pts: #top left corner of arrow bounding box
(cx, cy) = (pt[0]+w/2, pt[1]+h/2) #center of the arrow
tips.append((int(cx - RADIUS*math.cos(rad)), int(cy + RADIUS*math.sin(rad))))
return tips
In the above code,
matches
is the list generated by match_and_picke()
deg
pt
w
h
RADIUS
tips
Plotting the list of \((px, py)\) on the original image will give us something like this:
Again, if you observe closely, there are multiple points per arrow.
To produce our final output, coalescing multiple points into one final point is straightforward: if two points \(a\) and \(b\) are within a certain distance, then \(b\) is a duplicate.
def coalesce(tips):
""" Reduces points close to together in tips into one point.
Returns the list of reduced points.
"""
thresh = 7
def not_distinct(p1, p2):
(p1x, p1y) = p1
(p2x, p2y) = p2
return (p1x-p2x)**2 + (p1y-p2y)**2 <= thresh**2
def update(pt, distinct_pts):
for dpt in distinct_pts:
if not_distinct(pt, dpt):
return
distinct_pts.append(pt)
return
if len(tips) <= 1:
return tips
distinct_pts = []
for pt in tips:
update(pt, distinct_pts)
return distinct_pts
Here, the input tips
is the list of \((px, py)\) for all the arrows in an image. thresh
is the pixel distance between two points for them to be considered duplicates. The return value distinct_points
is then the final set of points in an image, with one point per arrow. This algorithm runs in \(O(n^2)\).
One may imagine that coalescing can be improved to \(O(n)\) if we first sort the input list and use the ‘runner technique’. Then we would only need a ‘fast’ pointer to look ahead and find the next distinct point, then add the ‘slow’ pointer to our output and jump the ‘slow’ pointer to the ‘fast’ pointer. But this requires the precondition that no pair of arrows point to similar \(x\) or \(y\) coordinates, which is something we cannot guarantee.
The following function transform_and_coalesce()
is a wrapper that performs transform()
and coalesce()
on data loaded from a pickle file and then writes the final output into a csv file. The first line is the number of points in the image, followed by one point per line.
def transform_and_coalesce(img_name):
""" Applies transform() and coalesce() on data from the pickle file
of img_name. E.g. M21.pkl from PKL_DIR
Saves output as a csv file. E.g. M21.csv
"""
print('Transforming for image %s' %img_name)
pkl_name = "".join(img_name.split('.')[:-1])+'.pkl' #eg M21.pkl
try:
with open(os.path.join(PKL_DIR, pkl_name),'r') as f:
matches = pickle.load(f)
except IOError:
print('Failed to transform img %s. pkl file not found.' %img_name)
return
tips = coalesce(transform(matches))
#Write into csv
csv_name = ''.join(img_name.split('.')[:-1])+'.csv'
with open(os.path.join(LOC_DIR, csv_name),'w') as f:
wr = csv.writer(f)
wr.writerow([len(tips)]) #1st line is number of arrows
for pt in tips:
wr.writerow(pt)
return
To make sure we get the correct result plot()
draws the points on the image by loading the points from the csv file we created.
def plot(img_name, output_path='plots'):
print('Plotting image %s' %img_name)
img_rgb = cv2.imread(os.path.join(IMG_DIR, img_name))
csv_name = "".join(img_name.split('.')[:-1])+'.csv' #eg M21.pkl
try:
with open(os.path.join(LOC_DIR, csv_name),'r') as f:
rd = csv.reader(f)
num = int(rd.next()[0])
for i in range(num):
pt_str = rd.next()
pt = (int(pt_str[0]), int(pt_str[1]))
cv2.drawMarker(img_rgb, pt, CI.next(), cv2.MARKER_CROSS, 20, 4)
cv2.putText(img_rgb, str(num), (10,800), cv2.FONT_HERSHEY_SIMPLEX, 2,(0,0,255),2,cv2.LINE_AA)
except IOError:
print('Failed to plot img %s. pkl file not found.' %img_name)
return
out_name = ''.join(img_name.split('.')[:-1])+'_p.jpg'
out = os.path.join(output_path, out_name)
cv2.imwrite(out, img_rgb) #eg M21_p.jpg
Here is an example of the final result:
We can see that we now only have one point for each arrow, and we are done!
The full code complete with some additional functions for bulk processing are at the end of this page.
There are some design choices in my program that I would like to address.
First, it is possible, and indeed more efficient, to do away with the pickling and directly pass the output of match()
into transform_and_coalesce()
. However, I chose to have an intermediate pickling stage for the sake of easier debugging. It allows me to draw plots from the pickled data to make sure match()
works as it should. It also allows me to test transform_and_coalesce()
without needing to run match()
over and over again.
Second, my supervisor asked me why I have individual final csv files for each image, instead of saving the data for all my images in one big csv. The answer is that I need the data for machine learning applications later on, but I will probably only need data for a few images at any given time. So separating the csv’s mean less slicing and dicing later on.
Working on this image processing project was something I was really excited about. Through this process, I came to realize what motivates me. I am excited by stuff that I don’t know how to do, stuff that I feel completely clueless about. At times, work goes like this:
On the other hand, I’m less excited about problems where I know there are standard tools and a standard solutions, because then it is grunt work, and there is less to learn from it.
The more clueless I am, the more I want to get my hands on the problem. It is beyond the excitement of learning a new technology or skill. It is the passion of doing something where there is no known recipe, and trying to find out if it can even be done!
import numpy as np
import cv2
import os
from scipy import misc, ndimage
from multiprocessing import Pool
import pickle
import math
import re
import csv
IMG_DIR = {path to original images}
STRIPPED_DIR = {path to save stripped images}
TMPL_DIR = {path to templates}
MASK_DIR = {path to masks}
PKL_DIR = {path to pickle files}
LOC_DIR = {path to final csv files}
GREEN = np.array([89, 248, 89])
MATCH_THRESH = 11
MATCH_RES = 1 #specifies degree-interval at which to match
#Match thresholds and resolution were empirically tuned
RADIUS = 42 #Half the length of the arrow in pixels
class Color_Iterator(object):
colors = [(0,0,255), (255,0,0), (204,204,0)]
i = 0
def next(self):
self.i += 1
return self.colors[self.i % len(self.colors)]
CI = Color_Iterator()
def strip(img_name):
""" Removes background from img_name in IMG_DIR, leaving only green arrows.
Saves stripped image in STRIPPED_DIR, as img_name's'.jpg
"""
print('Stripping img %s' %img_name)
arr = misc.imread(os.path.join(IMG_DIR, img_name))
(x_size, y_size, z_size) = arr.shape
for x in range(x_size):
for y in range(y_size):
if not np.array_equal(arr[x, y], GREEN):
arr[x, y] = np.array([0, 0, 0])
img_name = "".join(img_name.split('.')[:-1])
misc.imsave(os.path.join(STRIPPED_DIR, img_name+'_s.jpg'), arr) #eg M21_s.jpg
return
def strip_all(num_processes=2):
""" Applies strip() to images of name M{start..start+num_images-1}.jpg.
This method uses multiprocessing:
num_processes -- the number of parallel processes to spawn for this task.
(default 2)
"""
imgs = [i for i in os.listdir(IMG_DIR) if re.match(r'M[0-9]*.jpg', i)]
print('Stripping background from %d images' %len(imgs))
pool = Pool(num_processes)
pool.map(strip, imgs)
pool.close()
pool.join()
print('Done')
return
def make_templates(base='base_short.png'):
""" Makes templates for rotational-deg=0...359 from base in TMPL_DIR.
Saves rotated templates as tmpl{deg}.png in TMPL_DIR
"""
try:
base = misc.imread(os.path.join(TMPL_DIR, base))
except IOError:
print('Failed to make templates. Base template is not found')
return
for deg in range(360):
tmpl = ndimage.rotate(base, deg)
misc.imsave(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg), tmpl)
return
def make_masks():
""" Makes masks from tmpl{0...359}.png in TMPL_DIR.
Saves masks as mask{0...359}.png in MASK_DIR
"""
for deg in range(360):
tmpl = cv2.imread(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg),
cv2.IMREAD_GRAYSCALE)
if tmpl is None:
print('Failed to make mask {0}. tmpl{0}.png is not found.'.
format(deg))
else:
ret2, mask = cv2.threshold(tmpl, 0, 255,
cv2.THRESH_BINARY+cv2.THRESH_OTSU)
cv2.imwrite(os.path.join(MASK_DIR, 'mask%d.png' %deg), mask)
return
def match_and_pickle(img_name):
""" Apply template matching to img_name in IMG_DIR, using
tmpl{0...359}.png and mask{0...359}.png.
Output: Coordinates of the top left corner of matching templates,
in the format [(deg, [points])], saved in img_name.pkl.
"""
print('Matching img %s' %img_name)
img_rgb = cv2.imread(os.path.join(STRIPPED_DIR, img_name))
img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
matches = []
for deg in range(0, 360, MATCH_RES):
tmpl = cv2.imread(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg),
cv2.IMREAD_GRAYSCALE)
mask = cv2.imread(os.path.join(MASK_DIR, 'mask%d.png' %deg),
cv2.IMREAD_GRAYSCALE)
if tmpl is None or mask is None:
print('Failed to match for tmpl %d.' %deg)
else:
w, h = tmpl.shape[::-1]
res = cv2.matchTemplate(img_gray, tmpl, cv2.TM_SQDIFF, mask=mask)
loc = np.where(res < MATCH_THRESH)
pts = zip(*loc[::-1]) #top left corners of arrow bounding box
if len(pts)>0:
matches.append((deg, pts))
file_name = os.path.join(PKL_DIR,
"".join(img_name.split('_')[:-1])+'.pkl') #eg M21.pkl
with open(file_name,'wb') as f:
pickle.dump(matches, f)
return
def match_all(num_processes=2):
""" Applies match() to images of name M{start ... start+num_images-1}s.jpg.
This method uses multiprocessing:
num_processes -- the number of parallel processes to spawn for this task.
(default 2)
"""
imgs = [i for i in os.listdir(STRIPPED_DIR) if re.match(r'M[0-9]*_s.jpg', i)]
print('Matching %d images' %len(imgs))
pool = Pool(num_processes)
pool.map(match_and_pickle, imgs)
pool.close()
pool.join()
print('Done')
return
def draw_from_pkl(img_name, output_path='./'):
""" Draws bounding boxes on stripped ver of img_name with matches from
img_name.pkl.
Saves output image in output_path.
Example: draw_from_pkl(M21.jpg, './') draws boxes on M21_s.jpg, and
saves output as M21_r.jpg in the current directory.
"""
img_name = ''.join(img_name.split('.')[:-1])+'_s.jpg'
img_rgb = cv2.imread(os.path.join(STRIPPED_DIR, img_name))
pkl_name = os.path.join(PKL_DIR,
"".join(img_name.split('_')[:-1])+'.pkl') #eg M21.pkl
with open(pkl_name,'r') as f:
matches = pickle.load(f)
for (deg, pts) in matches:
tmpl = cv2.imread(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg),
cv2.IMREAD_GRAYSCALE)
w, h = tmpl.shape[::-1]
for pt in pts:
cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), CI.next(), 1)
img_name = "".join(img_name.split('_')[:-1])
cv2.imwrite(os.path.join(output_path, img_name+'_r.jpg'), img_rgb) #eg M21_r.jpg
def transform(matches):
""" Transforms a list of (deg * coordinate list), where the coordinates
are of the top-left corner of the tmpl bounding boxes, to a list containting
the coordinates of the tips of the arrows.
Returns the list of transformed coordinates.
"""
tips = []
for (deg, pts) in matches:
tmpl = cv2.imread(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg), 0)
w, h = tmpl.shape[::-1]
rad = math.radians(deg)
for pt in pts: #top left corner of arrow bounding box
(cx, cy) = (pt[0]+w/2, pt[1]+h/2) #center of the arrow
tips.append((int(cx - RADIUS*math.cos(rad)), int(cy + RADIUS*math.sin(rad))))
return tips
def coalesce(tips):
""" Reduces points close to together in tips into one point.
Returns the list of reduced points.
"""
thresh = 7
def not_distinct(p1, p2):
(p1x, p1y) = p1
(p2x, p2y) = p2
return (p1x-p2x)**2 + (p1y-p2y)**2 <= thresh**2
def update(pt, distinct_pts):
for dpt in distinct_pts:
if not_distinct(pt, dpt):
return
distinct_pts.append(pt)
return
if len(tips) <= 1:
return tips
distinct_pts = []
for pt in tips:
update(pt, distinct_pts)
return distinct_pts
def transform_and_coalesce(img_name):
""" Applies transform() and coalesce() on data from the pickle file
of img_name. E.g. M21.pkl from PKL_DIR
Saves output as a csv file. E.g. M21.csv
"""
print('Transforming for image %s' %img_name)
pkl_name = "".join(img_name.split('.')[:-1])+'.pkl' #eg M21.pkl
try:
with open(os.path.join(PKL_DIR, pkl_name),'r') as f:
matches = pickle.load(f)
except IOError:
print('Failed to transform img %s. pkl file not found.' %img_name)
return
tips = coalesce(transform(matches))
#Write into csv
csv_name = ''.join(img_name.split('.')[:-1])+'.csv'
with open(os.path.join(LOC_DIR, csv_name),'w') as f:
wr = csv.writer(f)
wr.writerow([len(tips)]) #1st line is number of arrows
for pt in tips:
wr.writerow(pt)
return
def transform_and_coalesce_all(num_processes=2):
"""Applies transform_and_coalesce() to images of name
M{start ... start+num_images-1}s.jpg.
This method uses multiprocessing:
num_processes -- the number of parallel processes to spawn for this task.
(default 2)
"""
imgs = [i for i in os.listdir(IMG_DIR) if re.match(r'M[0-9]*.jpg', i)]
print('Transforming %d images' %len(imgs))
pool = Pool(num_processes)
pool.map(transform_and_coalesce, imgs)
pool.close()
pool.join()
print('Done')
return
def plot(img_name, output_path='plots'):
print('Plotting image %s' %img_name)
img_rgb = cv2.imread(os.path.join(IMG_DIR, img_name))
csv_name = "".join(img_name.split('.')[:-1])+'.csv' #eg M21.pkl
try:
with open(os.path.join(LOC_DIR, csv_name),'r') as f:
rd = csv.reader(f)
num = int(rd.next()[0])
for i in range(num):
pt_str = rd.next()
pt = (int(pt_str[0]), int(pt_str[1]))
cv2.drawMarker(img_rgb, pt, CI.next(), cv2.MARKER_CROSS, 20, 4)
cv2.putText(img_rgb, str(num), (10,800), cv2.FONT_HERSHEY_SIMPLEX, 2,(0,0,255),2,cv2.LINE_AA)
except IOError:
print('Failed to plot img %s. pkl file not found.' %img_name)
return
out_name = ''.join(img_name.split('.')[:-1])+'_p.jpg'
out = os.path.join(output_path, out_name)
cv2.imwrite(out, img_rgb) #eg M21_p.jpg
def plot_label(img_name):
print('Plotting image %s' %img_name)
csv_name = "".join(img_name.split('.')[:-1])+'.csv' #eg M21.pkl
img_name = ''.join(img_name.split('.')[:-1])+'_s.jpg'
img_rgb = cv2.imread(os.path.join(STRIPPED_DIR, img_name))
try:
with open(os.path.join(LOC_DIR, csv_name),'r') as f:
rd = csv.reader(f)
num = int(rd.next()[0])
for i in range(num):
pt_str = rd.next()
(x, y) = (int(pt_str[0]), int(pt_str[1]))
cv2.drawMarker(img_rgb, (x, y), (0,0,255), cv2.MARKER_CROSS, 20, 4)
cv2.putText(img_rgb, str((x, y)), (x+10, y+70), cv2.FONT_HERSHEY_SIMPLEX, 1,(0,0,255),2,cv2.LINE_AA)
cv2.putText(img_rgb, str(num), (10,800), cv2.FONT_HERSHEY_SIMPLEX, 2,(0,0,255),2,cv2.LINE_AA)
except IOError:
print('Failed to plot img %s. pkl file not found.' %img_name)
return
out = ''.join(img_name.split('_')[:-1])+'_p.jpg'
cv2.imwrite(out, img_rgb) #eg M21_p.jpg
def plot_all(num_processes=2):
imgs = [i for i in os.listdir(IMG_DIR) if re.match(r'M[0-9]*.jpg', i)]
print('Plotting %d images' %len(imgs))
pool = Pool(num_processes)
pool.map(plot, imgs)
pool.close()
pool.join()
print('Done')
return
After my first task of developing the Django web application at my summer internship, my second project, to my great excitement, is a lot more algorithmically involved. It involves processing raw data annotations into a format suitable for training Machine Learning algorithms.
Here’s the preamble. My research group is working on training a neural network to detect mitosis (cell division) in breast cancer histological images (See ICPR 2012 Mitosis Detection Contest). Apparently, mitosis count is a good measurement of the aggressiveness of cancer tumors. But mitosis detection is a very challenging task for a computer, because of its indistinctness and its variability. To the untrained eye, such as mine, a cell undergoing mitosis also looks pretty much the same as a regular cell. There are also four stages of mitosis, each with a different shape configuration. My future work involves improving our mitosis detection model. Exciting, but that is a job for another day.
Equally awesome is my current undertaking. Since this is another collaboration between IHPC and one of Singapore’s hospitals, we are training our neural net using data provided by our partner. However, the data was annotated in a very raw format, in the form of these:
Training data in its original form
The pain was two-fold. One, individual cells used for training must be cropped by hand. Two, the neural net would output of a set of pixel coordinates, and having no coordinate information, it was again up to the human eye to compare it to the images with green arrows.
I’m thus armed with the following goal:
I absolutely love this problem for two reasons: One, it’s meaningful as it drastically reduces human workload. Two, it’s a problem where I have no frickin’ clue how to solve, which in other words, mean it’s the best chance to learn something new!
It is indeed trivial to find the color green in the images. But I need to know where the arrows are pointing, which requires knowing their orientation. I soon realized this is a lot more involved than finding green. Also, I happen to have zero computer vision experience. However, having no clue is what inspires me to tackle a problem, and I quickly set to work figuring out a line of attack.
My first thought was to use a neural net, which I will train to give me the locations of arrow tips. But this felt like a bad solution because it requires a lot of knob-tuning, and it’s a black box.
Computer Vision whizzes on Quora pointed me in the right direction: Template Matching! Initially, I was thrown off by the fact that my arrows had different orientations. Did I have to implement a method that was rotationally-invariant? (Kim & Araujo, 2007) In the end, I went with a simple solution that avoided such overkill. I had 360 templates, one for each degree of rotation, and use the template matching methods in openCV.
This would help me place bounding boxes on arrows, as well as the rotation information of the arrows. With a bit of trigonometry, I can then extract information about the actual location the arrows are pointing.
My template matching algorithm is summarized as so:
Step 1 is trivial, as all the images had the same shade of green arrows. All I had to do was zero the non-green vectors in the RGBA matrix. I also did some parallel processing to speed this up for bulk processing of images (see complete code at bottom of page). Step 2 was likewise easy using SciPy, with this as my base template:
import numpy as np
import os
from scipy import misc, ndimage
IMG_DIR = {path to original images}
STRIPPED_DIR = {path to save stripped images}
TMPL_DIR = {path to templates}
GREEN = np.array([89, 248, 89])
def strip(img_name):
""" Removes background from img_name in IMG_DIR, leaving only green arrows.
Saves stripped image in STRIPPED_DIR, as img_name's'.jpg
"""
print('Stripping img %s' %img_name)
arr = misc.imread(os.path.join(IMG_DIR, img_name))
(x_size, y_size, z_size) = arr.shape
for x in range(x_size):
for y in range(y_size):
if not np.array_equal(arr[x, y], GREEN):
arr[x, y] = np.array([0, 0, 0])
img_name = "".join(img_name.split('.')[:-1])
misc.imsave(os.path.join(STRIPPED_DIR, img_name+'_s.jpg'), arr) #eg M21_s.jpg
return
def make_templates(base='base.png'):
""" Makes templates for rotational-deg=0...359 from base in TMPL_DIR.
Saves rotated templates as tmpl{deg}.png in TMPL_DIR
"""
base = misc.imread(os.path.join(TMPL_DIR, base))
for deg in range(360):
tmpl = ndimage.rotate(base, deg)
misc.imsave(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg), tmpl)
return
Next, following this wonderful tutorial, I had step 3 functioning in no time. The algorithm uses the cv2.TM_CCORR_NORMED
method on greyscaled images, with a match-acceptance threshold of >0.9
. It worked 99% of the time. But, in accordance with the power law of programming, that remaining 1% gave the biggest headache. Some edge cases gave me unacceptable false negatives:
False negatives (cv2.TM_CCORR_NORMED, acceptance threshold >0.9)
I could simply lower the threshold, but that gave me a ton of false positives before all the ground truths (ML lingo for the “real, actual positives”) were accepted. Due to the nature of the problem, it had to work 100%, and I needed a perfect solution.
I realized the problem was due to the proximity of the arrows, such that they would overlap with the back region of the templates during matching. The black background in the templates must be somehow excluded from the correlation.
To my dismay, openCV does not support alpha-dependent matching. As reluctant as I was to reinvent the wheel, I mentally prepared myself to write my own matching algorithm.
But then I stumbled upon an article about a new masking feature for openCV 3.2 in C. Aha! I couldn’t find useful information because there simply wasn’t documentation for openCV 3.2 for Python! Masks were my solution.
To create masks for each template, I used openCV’s thresholding feature, which converted my templates from grayscale to black and white. The black regions acted as the “opaque” mask, allowing the white “transparent” regions to be factored into the correlation calculations.
Example of a template after thresholding
Tuning the masks to work with openCV’s matchTemplate()
method was another issue for me. Although TM_CCORR_NORMED
now correctly identified my true positives, it gave me false positives that are even stronger than the true ones.
False positives (cv2.TM_CCORR_NORMED, acceptance threshold >0.9)
To this end, the wonderful folks on StackOverflow helped me tremendously. Using, the matching method TM_SQDIFF
and a new threshold of <11
(empirically tuned), and using png templates and masks instead of jpeg did the trick. Note that non-matches have correlation of >200
, so a acceptance threshold of 11 is really good.
Success! (cv2.TM_SQDIFF, acceptance threshold <11)
Cool was learning why TM_CCORR_NORMED
did not work with jpeg.
Because TM_CCORR_NORMED
is a normalized function with the denominator that it has, the black backgrounds meant I was dividing by 0 in many cases. This is not cool, but having a matrix entry of numpy.nan
wasn’t the issue because the comparison nan>threshold
yields false. The key was with the jpeg format. Because of lossy jpeg compression, entries that should have been 0 (black), were not. This gave me very small denominators, causing the correlation to blow up and give me false positives. This is why png masks, or using TM_SQDIFF
which does not have the denominator, fixes the problem.
Here is the end result and code, using TM_SQDIFF
and png masks:
import cv2
MASK_DIR = {path to masks}
RES_DIR = {path to save match results}
MATCH_THRESH = 11
MATCH_RES = 1 #specifies degree-interval at which to match
#Match thresholds and resolution were empirically tuned
def make_masks():
""" Makes masks from tmpl{0...359}.png in TMPL_DIR.
Saves masks as mask{0...359}.png in MASK_DIR
"""
for deg in range(360):
tmpl = cv2.imread(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg),
cv2.IMREAD_GRAYSCALE)
ret2, mask = cv2.threshold(tmpl, 0, 255,
cv2.THRESH_BINARY+cv2.THRESH_OTSU)
cv2.imwrite(os.path.join(MASK_DIR, 'mask%d.png' %deg), mask)
return
def match_and_draw(img_name):
""" Apply template matching to img_name in IMG_DIR, using
tmpl{0...359}.png and mask{0...359}.png.
Saves result with boxes drawn around matches as as img_name'r'.jpg in RES_DIR
"""
print('Matching img %s' %img_name)
img_rgb = cv2.imread(os.path.join(STRIPPED_DIR, img_name)) img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
for deg in range(0, 360, MATCH_RES):
tmpl = cv2.imread(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg),
cv2.IMREAD_GRAYSCALE)
mask = cv2.imread(os.path.join(MASK_DIR, 'mask%d.png' %deg),
cv2.IMREAD_GRAYSCALE)
w, h = tmpl.shape[::-1]
res = cv2.matchTemplate(img_gray, tmpl, cv2.TM_SQDIFF, mask=mask)
loc = np.where(res < MATCH_THRESH)
for pt in zip(*loc[::-1]):
cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0,0,255), 2)
print('Match for deg{}, pt({}, {}), sqdiff {}'.
format(deg,pt[0],pt[1],res[pt[1],pt[0]]))
img_name = "".join(img_name.split('_')[:-1])
cv2.imwrite(os.path.join(RES_DIR, img_name+'_r.jpg'), img_rgb) #eg M21_r.jpg
return
Again, a method for bulk processing of images is in the complete code at the bottom of this page.
Now, I have completed my first goal of locating all the arrows. Of course, the current code only finds and draws bounding boxes on images, which is useful for debugging purposes. It is easy to modify my code to extract the coordinates of the boxes, which tells me roughly where my arrows are (the top left corner of the boxes are given by the pt
variable in the code block above).
My next step is to transform these points, using the orientation information of the matching templates, into pixels that the arrows point to, which will be our ground truth. This doesn’t have to be exact of course, since we only need to tell our neural net roughly where to look, and if the ground-truth pixel is within the patch returned by the neural net.
Also, notice that the current program may output a couple of matching points for each arrow. I would need to coalesce these points into a single point for an accurate mitosis count. I will talk about my point transformation and coalescing algorithm in Part 2!
Below is the complete code for Part 1, with added error handling and print statements.
import os
import re
import cv2
import numpy as np
from scipy import misc, ndimage
from multiprocessing import Pool
IMG_DIR = {path to original images}
STRIPPED_DIR = {path to save stripped images}
TMPL_DIR = {path to templates}
MASK_DIR = {path to masks}
RES_DIR = {path to save match results}
GREEN = np.array([89, 248, 89])
MATCH_THRESH = 11
MATCH_RES = 1 #specifies degree-interval at which to match
#Match thresholds and resolution were empirically tuned
def strip(img_name):
""" Removes background from img_name in IMG_DIR, leaving only green arrows.
Saves stripped image in STRIPPED_DIR, as img_name's'.jpg """
print('Stripping img %s' %img_name)
try:
arr = misc.imread(os.path.join(IMG_DIR, img_name))
except IOError:
print('Failed to strip img %s. Image not found.' %img_name)
return
(x_size, y_size, z_size) = arr.shape
for x in range(x_size):
for y in range(y_size):
if not np.array_equal(arr[x, y], GREEN):
arr[x, y] = np.array([0, 0, 0])
img_name = "".join(img_name.split('.')[:-1])
misc.imsave(os.path.join(STRIPPED_DIR, img_name+'_s.jpg'), arr) #eg M21_s.jpg
return
def strip_all(num_processes=2):
""" Applies strip() to all images in IMG_DIR.
This method uses multiprocessing:
num_processes -- the number of parallel processes to spawn for this task.
(default 2)
"""
imgs = [i for i in os.listdir(IMG_DIR) if re.match(r'M[0-9]*.jpg', i)]
print('Stripping background from %d images' %len(imgs))
pool = Pool(num_processes)
pool.map(strip, imgs)
pool.close()
pool.join()
print('Done')
return
def make_templates():
""" Makes templates for rotational-deg=0...359 from base.jpg in TMPL_DIR.
Saves rotated templates as tmpl{deg}.png in TMPL_DIR
"""
try:
base = misc.imread(os.path.join(TMPL_DIR,'base_short.png'))
except IOError:
print('Failed to make templates. Base template is not found')
return
for deg in range(360):
tmpl = ndimage.rotate(base, deg)
misc.imsave(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg), tmpl)
return
def make_masks():
""" Makes masks from tmpl{0...359}.png in TMPL_DIR.
Saves masks as mask{0...359}.png in MASK_DIR
"""
for deg in range(360):
tmpl = cv2.imread(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg),
cv2.IMREAD_GRAYSCALE)
if tmpl is None:
print('Failed to make mask {0}. tmpl{0}.png is not found.'.
format(deg))
else:
ret2, mask = cv2.threshold(tmpl, 0, 255,
cv2.THRESH_BINARY+cv2.THRESH_OTSU)
cv2.imwrite(os.path.join(MASK_DIR, 'mask%d.png' %deg), mask)
return
def match_and_draw(img_name):
""" Applies template matching to img_name in IMG_DIR, using
tmpl{0...359}.png and mask{0...359}.png.
Saves result with boxes drawn around matches as as img_name'r'.jpg in RES_DIR
"""
print('Matching img %s' %img_name)
img_rgb = cv2.imread(os.path.join(STRIPPED_DIR, img_name))
if img_rgb is None:
print('Failed to match img %s. Image not found.' %img_name)
return
img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
for deg in range(0, 360, MATCH_RES):
tmpl = cv2.imread(os.path.join(TMPL_DIR, 'tmpl%d.png' %deg),
cv2.IMREAD_GRAYSCALE)
mask = cv2.imread(os.path.join(MASK_DIR, 'mask%d.png' %deg),
cv2.IMREAD_GRAYSCALE)
if tmpl is None or mask is None:
print('Failed to match for tmpl %d.' %deg)
else:
w, h = tmpl.shape[::-1]
res = cv2.matchTemplate(img_gray, tmpl, cv2.TM_SQDIFF, mask=mask)
loc = np.where(res < MATCH_THRESH)
for pt in zip(*loc[::-1]):
cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0,0,255), 2)
print('Match for deg{}, pt({}, {}), sqdiff {}'.
format(deg,pt[0],pt[1],res[pt[1],pt[0]]))
img_name = "".join(img_name.split('_')[:-1])
cv2.imwrite(os.path.join(RES_DIR, img_name+'_r.jpg'), img_rgb) #eg M21_r.jpg
return
def match_all(num_imgs, start=1, num_processes=2):
""" Applies match() to all images in STRIPPED_DIR.
This method uses multiprocessing:
num_processes -- the number of parallel processes to spawn for this task.
(default 2)
"""
imgs = [i for i in os.listdir(STRIPPED_DIR) if re.match(r'M[0-9]*_s.jpg', i)]
print('Matching %d images' %len(imgs))
pool = Pool(num_processes)
pool.map(match_and_draw, imgs)
pool.close()
pool.join()
print('Done')
return
This summer, I’m doing a research internship in sunny Singapore. In particular, I’m working at the Institute of High Performance Computing (IHPC), a research institute under the parent company that is the Agency for Science, Technology and Research, known as A*STAR.
To many who are probably unfamiliar, A*STAR “is Singapore’s lead public sector agency that spearheads economic-oriented research to advance scientific discovery and develop innovative technology”. It’s a pretty cool environment, and I am working on some machine learning projects with my supervisor.
As an AI Research Intern, the projects in which I’m involved are mostly about automating patient diagnosis through Machine Learning and Computer Vision.
My first task this summer was to create a ‘proof-of-concept’ web application that would demonstrate to hospitals the use of a lung cancer identification algorithm our team developed. This algorithm is a neural net that takes in a set of DICOM images. A dicom image (Digital Imaging and Communications in Medicine) is the standard output format for X-Ray and CAT scans. An image is 2-D slice, and a set of images from a CAT scan gives a 3-D rendering of, say, a lung. Our algorithm then spits out the xyz-coordinates of where the cancerous tumors might be located in said lung.
Tumor identification is traditionally done by the human eye, so it is hoped that our program can automate the process. What’s needed now is a web-based platform to serve our program to the customers. That is my job to build.
To my delight, there already exists an open-source medical image viewer in JavaScript lying around on GitHub. We fondly call it the Papaya Viewer. No need to reinvent the wheel then!
With a good overview of the problem, I could zoom in on the specifications. I need to develop a web-application that:
My web development being experience close to naught (this was before I built this website), I got to work looking up possible tools I could use. I knew it had to be dynamic application, so my choices included Ruby on Rails and Django. Since I fell in love with the sheer awesomeness of Python last semester writing OS projects (synchronization & network programming), I went with Django. (I might do a future post about why I think Python is great, and why it replaced Java as my language of choice)
The amazing quality of Django’s documentation got me up to speed quickly, and a day of tutorials and fiddling later, I was quickly building my first web application.
I’d show you how beautiful it looks, but in reality though, it’s a proof-of-concept, and minimally viable demo, so we’re wasting no time dressing it up. It’s only crucial that it works. Which it wonderfully does.
But most important to me personally, is that I learned the process of creating and deploying a web application, such as url management, utilizing modularity in HTML templates, and how MVC is applied in web development.
I also learned what a framework like Django was about. It’s like the inverse of an API, where instead of you calling the the API, the framework calls your code (see Inversion of Control). It’s an MVC pattern with the C already in place, and you only worry about the M and the V; a chef-for-hire where where you just provide the ingredients. It is awesome.
Django does seem a bit of an overkill for the present state of my application. After all, my site has only two pages, one for uploading, and one for viewing, which sounds like a couple of HTML and static files.
However, Django is made to be flexible and extensible. If I get the chance to return to this project, potential new features I want to add include a database to store patient data, visible to each authenticated user. And for that, Django is perfect.
]]>