exporting Mendeley tags to csv or xls

I recently was struggling with a data problem, one that I hadn’t anticipated being so … annoying. I eventually found a solution, and thought I would share it so others wouldn’t have to go through the same thing that I did. [Update: I have a link to the files that I made at the bottom of the post.]

One of the projects that I am working on is a huge meta-analysis of research studies surrounding simulations used for learning STEM content, inquiry, and related skills. I am leading a team of researchers in this task (some of whom are at a different institution), and initially tried to figure out the best way to collaborate in reading and coding thousands of article abstracts. I decided to go with Mendeley, mostly because I knew it would be easy to get the citations into it, it’s free, it works really well, and I could set up collaborative groups to facilitate our coding process. What I hadn’t anticipated would be how difficult it would be to get the citations out of Mendeley and into a spreadsheet (which I was using for analysis of the abstract data and the next steps of our overall project). Specifically, I needed to get the tags in Mendeley out, since we were using the tag field for our coding.

[Sidenote: Using tags to code the abstracts in Mendeley actually worked pretty well in my opinion.]

Mendeley, while awesome in many respects, does not have many options for exporting data. You can do BibTeX (.bib), RIS (.ris), or EndNote XML (.xml). Through some sleuthing, I managed to figure out that BibTeX was probably my best bet as the tags were definitely part of that export (they didn’t seem to be in the xml file and I didn’t really want to deal with RIS if I didn’t have to). Now, of course, the new problem is that BibTex isn’t a spreadsheet and it’s only readable by other citation management software (and there’s a good reason why I don’t use EndNote anymore). So I still needed to get it into a csv type format. Luckily, through some interwebz searching, I found the program JabRef, which is an open source citation manager and is able to export to csv. Hooray!

However, my problems still weren’t quite over, as the default csv export filter didn’t include the tags that I wanted. Although free, open source programs are great in a lot of ways, they are usually not so great in the documentation of certain operations. And this was one of those situations.

So here’s how to make a custom csv export filter for JabRef that includes the tags that Mendeley uses. [Update: you can skip this step and just download the files I made – links at the bottom of the post.]

First, download the src files. This includes the the standard export filter layouts that you can then edit and tweak to your heart’s content. They can be found here: Open up the file openoffice-csv.layout (in your text editor of choice) and decide which fields you want to be included in your export to csv. “\mendeley-tags” is the correct way to refer to the Mendeley tags in the BibTeX file. My suggestion is to definitely delete the “\annote” field (especially, if like me, you had a bunch of random crap in the Note field in Mendeley, some of which took up a lot of space). You can of course change the order of these and delete other fields as well. After going through the rest of this process, I would have taken out more fields due to the regular expressions weirdness that came out the other side when there were strange symbols or characters (which happened a lot in the author and abstract fields – more on that later). If you don’t need the address, I would suggest deleting that since there were a bunch of formatting issues with it in the csv version.

You can also edit the openoffice-csv.begin.layout file to change the headings for the csv export – and obviously, it’s easier later on if you have ensured that the data and the headings match up. :) Make sure to rename your edited files (I called mine custom-csv.layout and custom-csv.begin.layout) and then save them somewhere convenient.

Then go back to JabRef, select Menu Options – Manage custom exports – Add new. Fill in the fields: export name – call it whatever you want; main layout file: navigate to your saved .layout file; file extension: .csv. Click Ok and Close. Then go to Menu – File – Export and your newly defined export filter should be listed there.

Now, this solution is not perfect by any means. There were definitely lots of formatting and regexp hiccups in the transformation and export processes. I’m not sure where these occurred and I’m not skilled enough at regexp to fix them. So, I had to go through my csv file and correct a lot of the issues (the concatenate function in Excel was exceedingly helpful in this regard). But, in the end I got all 2392 of my citations into an xls format and can now manipulate my data with ease.

[Update: Here are the files that I created. This is the .layout file and this is the begin.layout file.]

Remembering my uncle Dan

Today was my uncle Dan’s memorial service. He passed away last week, surrounded by his wife, two daughters, and two of his sisters. He was only 50 years old, but when he was originally diagnosed with cancer in 2001, his doctors didn’t think he’d make it to his 40th birthday. Some experimental surgeries by amazing doctors in DC, as well as a fighting spirit and supportive family and friends, made the last 11 years much more than any of us could have hoped for in 2001.

Celebrating Dan’s 50th Birthday

Five days before he passed, we took him to Disneyland for one last trip. Disneyland was an important place for our family, somewhere we frequently went growing up. He really wanted to go one last time, and even though a lot of us felt that it was not a great idea based on his condition (he was dependent on oxygen and not very strong) and the extreme heat that weekend, we went and had a nice and memorable time. And I was able to say goodbye.

One last memorable trip to Disneyland

He did not lose his battle with cancer. If anything, he won. The odds were stacked heavily against him, and yet he fought and suffered and fought some more. He had 11 amazing years with his family, going on trips and making lasting memories. He had 11 years to watch his daughters grow up into beautiful strong young women. He had 11 more years to find fulfillment through his work as a physical therapist. He had 11 years to research, write, and publish a novel based on one of his passions – the U.S. space program (it’s a hypothetical based on the scenario that we could have rescued the Columbia crew and it is a great read).

He was stronger than I can possibly imagine and stronger than anyone should have to be. Yet, he was always joking and smiling and keeping a positive attitude, even up until the end.

I will remember his sense of humor, his love of the space program, and his amazing strength. He will not be forgotten.

Dan with a spoon on his nose

Advice for Grant Writers

I have written or helped write a lot of grants (like >10) in the last two years of my academic career. I have also recently gone to NSF and reviewed proposals. And people keep asking me if I have any advice for them when they are writing grants. So I thought I would put together something that might be helpful. Some of this advice is just general writing advice, but I think it is especially important or relevant for grant writing.

[caveats: I am putting this list together from the perspective of someone (me) who is relatively new to grant writing and still very early in her career and who therefore is approaching this as a new form of writing. I mostly write grants to go to NSF in the area of STEM education research, so this advice is based on my experience with funding through that organization.]

Read the call/request for proposals very carefully

This may seem like an easy one, but it’s really important. The tone and details of the request/call for proposals can tell you a lot about not just what kinds of projects the program or agency is interested in, but also how you should frame your argument for funding your project. Also, sometimes there are very specific requirements, such as asking you to identity the type and/or size of the proposed project in the first sentence or the title. These types of seemingly minor mistakes sometimes make it difficult for reviewers to properly situate your proposal and can lead people to think you won’t be able to pay attention to details.

What’s the big idea?

This is perhaps the most important part of this process. What is the big idea you want to address? What is the big problem you are trying to solve (and how will you solve it)? Once you have figured this out, you need to create a story or an argument for (a) why this is a big idea/problem, (b) why this is the best way to solve/fix the problem, and (c) why you are the team to do it. These three points all need to be addressed early and often in the proposal. You need to convince the reviewers that this is important and innovative work that is achievable.

Talk to a program officer

Two important things to remember: (1) program officers (at least at NSF), as federal employees, are required by law to respond to your emails and phone calls, and (2) program officers want you to submit the best proposal you can because it’s better for everyone if they are funding innovative and successful projects. This means that you should not hesitate to contact them about your initial ideas to make sure that your proposal fits within the program boundaries. Program officers are usually good about giving you advice about which program would be a good fit for your project and whether or not it is missing important pieces. Of course, keep in mind that there are limits to the type and amount of guidance they can give you at the proposal stage.

With a little help from your friends

Having a third party read your proposal is essential as well, especially if the proposal is about a topic that you have been immersed in for a while. You have probably developed some shortcuts, slang, or ways of talking about this topic that are not immediately apparent to someone who hasn’t been inside your brain recently. A third party, even someone who is familiar with your work, can help bring an outside view or perspective (like someone who might be reviewing your proposal) that should clarify unclear concepts or phrases, help you structure your argument effectively, and ensure that you have properly explained what you’re planning on doing, how you will do it, and how your team is well-suited to perform the work.

Make it easy for your reviewers

Things that reviewers are looking for specifically: overall big idea, prior work/results related to this project, intellectual merit, broader impacts, prior NSF support (or the equivalent), team, and workplan. If you write your proposal in such a way that it is difficult for a typical reviewer to find any one of these pieces, it is not going to help your case. A good proposal is clear, as concise as possible, and complete. Note that the prior NSF support section should include not just that you were part of a funded project but also any major findings of that project. Additionally, there is a fun catch-22 with funding. In order to get funding, you (usually) need to have shown that you have already been successfully funded. This means that if you’re an early career researcher and applying for your first grants, it is extremely important to find more senior researchers to team up with who have a history with that funding agency. A well-known advisory board also helps in this regard.

All of the pieces

There are a LOT of documents and details to keep track of during the proposal writing process. It is extremely important that you stay organized throughout, otherwise the few days before the deadline are going to be hellish (I’m not kidding). Make sure you start the process soon enough to ask for appropriate letters of support from your advisory board and collaborators. Make sure that all of the details, language, and formatting match across the narrative, summary, references, budget, budget justification, letters, subaward information (if relevant), current & pending, biosketches, data management plan, and mentoring plan. Make sure you don’t forget any of those pieces. Make sure that you review the final pdf before it gets submitted – especially if you have a lot of figures or diagrams (sometimes certain programs don’t render these correctly when converting to pdf).

Sign up to review

A lot of young researchers seem unsure about this. But basically, if you have finished your Ph.D., you can review for NSF. They have some advice for how to do this buried on their website somewhere, but the easiest thing to do is to create a short version of your CV (about 2 pages) that highlights the kind of work that you do and your areas of expertise and send it to a program officer in a relevant area, letting him or her know that you are interested in volunteering to be on a review panel. If that specific program officer doesn’t need someone with your expertise for their panels, they usually pass along your information to their colleagues to see if someone else does. Being on a review panel was one of the most interesting and valuable experiences I have had in terms of learning how to write grant proposals and discuss research projects. You get to see things from the other side, learn about lots of cutting edge research and ideas, meet people in related fields, and most importantly, see what parts of the proposal are most important during those funding conversations.

Keep trying

In general, the world of academia and research includes a lot of rejection: rejected conference proposals, rejected journal articles, and unfunded research proposals. Research proposals are notoriously hard to get funded. The acceptance rates at NSF are in the single digits in some programs (overall, NSF’s acceptance rate is around 28%). All of this rejection requires a thick skin and an optimistic outlook. Whenever you get reviews back from a proposal, try to use the feedback constructively to either make improvements to your funded project or to write an even better proposal next time.

The Next Chapter

I am pleased to announce that I have accepted a research job with the Center for Technology and Learning at SRI International! I will be starting in mid-July and can’t wait to start working on some of the projects there and continue building my research program. I think that SRI will be a great place for me and allow me to take advantage of my unique background and training in order to do really innovative work in the intersection of science education, technology, and measurement. SRI supports research in all sorts of STEM areas, not just education, and there are a ton of resources available.

SRI is located in Menlo Park, a few blocks away from Stanford (the “S” in SRI used to refer to Stanford – although they haven’t been affiliated with them since 1970). I am excited to move back to California (but I think my family is even more excited) and it should be interesting living up in the Bay Area instead of SoCal like I’m used to.


I would also like to say that I was lucky enough to have to make a decision about where I wanted to work next, and although my other option was very exciting in a lot of ways (due to its location – in Singapore – and research opportunities), I feel like SRI will be a really good fit for the next chapter of my career.

The Beatles & Mad Men

*spoiler alert if you’re not caught up on Mad Men*

Mad Men is a great show. The writing is pretty fantastic and the characters are interesting and engaging. But also, the time period (the 1960s) was a time of great change in the U.S. One of my favorite things about the 60s is the music. Especially the Beatles. The Beatles produced a huge amount of extremely good music in a very short amount of time, a pace that is basically unheard of today. But it wasn’t just the amount of music or the quality of it but also how the music changed during that short period (1963-1970). To go from “I Wanna Hold Your Hand” to “I Want You (She’s So Heavy)” in about six years is pretty incredible. Popular music changed dramatically in that time, and even if the Beatles weren’t directly responsible for it, they were definitely the poster band for this change.

The last episode of Mad Men takes place in the summer of 1966 and the Beatles’ Revolver had recently come out. The Beatles had been mentioned many times in the episode (and in a few previous episodes). One of the clients wanted to basically remake the famous chase scene of Hard Days’ Night (a great movie) and needed a non-Beatles song to go in the ad. There was discussion about how it was impossible to get the Beatles to agree to be in a commercial. And it was also clear that Don and the clients still had an idea of what the Beatles sounded like that was still stuck in 1963-1964 (Don, after all, had seen them in Shea Stadium with his daughter when they first came over). They were slowly losing their grasp on what was currently going on in popular culture and weren’t able to keep up. At the end of the episode, Megan tries to help Don by giving him Revolver and tells him to listen to a certain track. And then…amazingly enough… “Tomorrow Never Knows” is playing on my TV. I was kind of shocked that they were able to (a) get the rights to it and (b) afford it. Because I’m sure it wasn’t cheap [I heard some estimates that it was probably around $250,000]. Sure enough, Don can’t stand to listen to the entire song, foreshadowing (I think) his and his generation’s increasing distance from the quickly changing popular culture.

Revolver is one of my favorite Beatles albums. I was reading some stuff on the episode and people inevitably started arguing about what the best Beatles album is. There are four main front-runners: Abbey Road, Revolver, Sgt. Pepper’s, and the White Album (although I’m sure most of those who argue for the White Album realize that half of it is not super amazing). I have been firmly in the Abbey Road camp for some time. But the discussion of Revolver (and listening to it again on Monday) has made me rethink that just a little bit. I think probably I would say they are in a statistical dead heat, with Abbey Road slightly (but not statistically) ahead [nerd alert]. I have never been a huge fan of Sgt. Pepper’s as a whole, even though I really like some of the songs on it. I love the White Album a lot, but there are some days when I wish half of the songs on it didn’t exist so I wouldn’t have to keep skipping them (I’m thinking mostly of “Glass Onion”, “Long, Long, Long”, and “Mother Nature’s Son”). I also really like Let It Be, but most of those songs are just pretty good and not great (with the obvious exception of “Let It Be”).

All of this also started making me re-think one of my favorite lists to make: favorite Beatles songs. So, here is my current list of my favorite Beatles songs, in alphabetical order (because it’s way way too hard to rank them sufficiently):

  • All You Need is Love
  • A Day in the Life
  • Eleanor Rigby
  • The End
  • Everybody’s Got Something to Hide Except for Me and My Monkey
  • Got to Get you Into My Life
  • Happiness is a Warm Gun
  • Helter Skelter
  • Here, There and Everywhere
  • Hey Jude
  • Honey Pie
  • I Should Have Known Better
  • I Wanna Hold Your Hand
  • I Want You (She’s So Heavy)
  • I Will
  • I’ll Be Back
  • If I Fell
  • In My Life
  • Let It Be
  • Norwegian Wood (This Bird Has Flown)
  • Oh! Darling
  • Penny Lane
  • Revolution 1
  • Rocky Raccoon
  • Something
  • Strawberry Fields Forever
  • Tomorrow Never Knows
  • Twist and Shout
  • Two of Us
  • We Can Work it Out
  • While My Guitar Gently Weeps
  • Why Don’t We Do It in the Road?
  • With a Little Help From My Friends
  • You Never Give Me Your Money

some thoughts on AERA 2012

I just got back from AERA 2012, which was in Vancouver, British Columbia during the past week. It was my fourth AERA, and I hadn’t attended last year because I had basically sworn off the conference and decided I wasn’t going to go very frequenly anymore. Luckily, I was invited to a session at NCME (the sister conference to AERA that was co-occurring) and decided to go (it also didn’t hurt that the conference was in Vancouver). It turned out to be a great conference all around, and I went to a few good sessions and met with lots of old and new friends and colleagues.

The best thing about AERA usually is that everyone goes to AERA. I heard that this year there were over 13,000 people registered. That’s a bit crazy. This huge zoo is people is usually one of the reasons that I don’t like AERA, but for some reason it didn’t seem so bad this time around. Perhaps it was the way the conference center centre and hotels were located/configured, which forced people to walk around and run into people more than in other places. Or maybe it was that many of us chose not to get international cell/data packages and were reliant on congregating in certain places having wifi (and then consequently spreading around the passwords for the wifi). Whatever it was, I ended up running into lots of people that I wanted to talk to and had lots of interesting discussions.

I think also that one difference this time around was that I felt that for the first time I was able to help start connecting other people together, instead of relying on my advisors and professors to connect me to people they knew. That type of thing still happened, of course, and is an essential part of academic networking, but I was taking on that role for other, younger researchers now. And that felt kind of nice.

One particularly interesting set of conversations centered around using games in classrooms, how students might be viewing educational games within the context of a classroom setting, and how that context might (a) change how the student views the game/simulation and/or (b) change the type and amount of learning that is possible in that game (versus an out-of-school, informal, or student-directed session). This could possibly be highly dependent on what educational value the student might attach to the game outside of the classroom in the first place. For example, a commercial game being used in the classroom might have a larger “learning differential” than a game specifically designed to be used for learning (either informally or in a classroom). Definitely something to think some more about.

Vancouver was also a great city to visit. We had wonderful weather almost the entire time, which just highlighted how beautiful the city is. There was lots of great food to be eaten, especially seafood (yum!).

On my last afternoon, I (along with some others) went to the Vancouver Aquarium. It was a really nice facility and I would totally recommend going there if you are visiting and you like awesome things. Here are a couple pictures of the aquarium and Vancouver. More pictures will be posted soon (somewhere).


Part of the Vancouver skyline, from Stanley Park.


Awesome jellyfish at the Vancouver Aquarium.


Part of the sea lion feeding/training non-show.


Baby clownfish. Awww.


Totem pole exhibit in Stanley Park.


Watching sea planes land from the Convention Centre.

[side note: all of these pictures were taken with my iPhone 4 because (a) it’s what I had with me the entire time and (b) my better camera just broke.]

Don’t believe everything you hear

Yesterday I saw this BBC video about the McGurk effect. Basically, it’s a phenomenon where your brain interprets sounds differently based on what you are seeing. It’s a really amazing and robust effect, especially since it is not dependent on you not knowing about it.

This reminds me a lot of when I was teaching undergraduate physics labs and we would do the famous ball drop experiment where you would release two balls (same size but different mass) from the same height and see which one hits the ground first. The majority of these intro students would usually predict that the more massive ball would hit the ground first (a common intuition to have). The interesting thing was that a few of these students would still think this after we had done the experiment.

I remember asking some of them why they still thought that the more massive ball hit the ground first. And inevitably, some of them would say, “I heard it hit first.” At the time, this seemed kind of crazy to me because I had been there and heard the two balls hit at the same time and other people in their group had also heard them hit together. But they insisted that they heard a separation.

Later when I started studying science education I learned the name for this phenomenon: theory-laden observations. The students thought that the more massive ball would hit first and this influenced how they took in information and therefore their observation confirmed this idea. However, it seems possible that it might be even more complicated than that. Although the McGurk effect seems primarily to be focused on hearing different phonemes and sounds based on lip movements, it’s possible that the parts of the brain being confused in that area are the same as the ball-drop phenomenon. Students *think* they see the massive ball hit first and therefore *hear* it hit first.


(via kottke)

last day in Hong Kong

I am leaving to go back to the states very early tomorrow morning, so today is my last full day here and of course it’s rainy and gross (although, I suppose I should be happy that it wasn’t like this the entire time). Luckily I had a bunch of work to do anywhere, so staying in my hotel room was kind of the general plan for most of the day anyway. (I decided against going to Ocean Park for many reasons, most of which were confirmed when I woke up still exhausted this morning and then also looked out the window.)

The last two days were filled with many awesome things which I will delve into more with future posts. Highlights include: going to Lantau Island (awesome cable car ride, giant Buddha statue, amazing view of some islands in the South China Sea), awesome (and inexpensive) sushi, amazing dim sum with the locals, watching the final horserace of the season with about 80,000 of my closest friends, and stores that sold only old school and/or hard to find LEGO mini-figs and other related geeky products. In short, it was a good weekend. I am also now really tired and looking forward to getting back to my normal level of exhaustion. :)

The conference was also really great and I was able to reconnect with old friends/colleagues and meet lots of new people and hear about all sorts of interesting projects and ideas that were floating around. This conference was especially great due to the great diversity of researchers, both in terms of intersecting and complementary fields of study but also in terms of backgrounds and countries and educational contexts. I think they said the conference had about 400 attendees from 38 different countries! Pretty cool.