RC3: Day 10

This is the last set of pictures from this experiment. I’m going to monitor the 99% d2o sample over the course of time to ensure there is no seed growth, and will update accordingly.

DDW3: Day 4

I have to admit how surprised I am that the arabidopsis seeds have grown so quickly. I guess this is why they are so useful for genetics experiments.

No pictures this weekend.

 

Web Analytic Tools for Impact Factor

I’m going to publish my web analytics data weekly on this site. WordPress has it’s own limited site stats. On top of that I linked my Google Analytics account to this notebook, and I’m looking to check out one more analytics software: either Piwik (which is a free open sourced software that you install on your server) or CrazyEgg (which was suggested to me by Alan Marnett of BenchFly fame). Ideally I’d like to do both, but the downfall of installing all these analytic software is that they may bog down the site and cause lengthy loading times which would drive away traffic. Maybe scientists are different and will wait to see content, but I don’t want to take that chance.

Why so many different analytics?

Well from what I can tell, the data that is generated isn’t consistent. I experienced this on my own personal blog which is hosted by Google (Blogger). There are site stats that are incorporated with the Blogger software itself and I initially setup Google Analytics to analyze this blog (before Blogger added analytics). Both sources reveal different information (hits, links, navigation, etc) and the strange thing is that both are Google Analytics. I can’t tell if they are different versions of Analytics or the same version, but the fact is that the same piece of software somehow generates two different results.

So far in my own experience here the same rule applies: Google Analytics tells me one thing and WordPress tells me another. So by adding at least one more piece of software I’ll hopefully be able to get a better account of how my notebook is being used by the public. After all having more data is better than not having enough!

I will publish various pieces of information from all sources like hits for the week, top visited page, relevant references, or whatever else I determine is useful to the cause. Since most of this data is presented to me as charts and graphs, I will upload those graphs here for you to enjoy as well!

This weeks should be particularly enjoyable since I published that ONS vs science infrastructure article. I bet you can’t wait to see the data!

 

Publishing Openly on the Web: Impact Factor

It occurred to me, sadly not too long ago, that the web is way different than a book. Traditionally in science the only way to gauge your presence besides having a network of colleagues is through peer review citations. By this I mean having your publications cited in other publications.

Back before computers I have no idea how people measured how many times they were cited. Did they even care? Were the publication places so few that it was easy? I wish I could ask Gilbert Lewis!

But now, this is rather easy thanks to all the search algorithms used around the internet.

Typically this method of measuring scientific worth is known as impact factor. While it has its uses (it is easy to gauge how prominent someone is if the number is high), it has several flaws and the worst of which is used for prejudging papers submitted for peer review.

Several publications (Science and Nature being the proclaimed best) accept papers for print based on perceived impact factor, meaning reviewers and editors judge papers based on whether or not they think a paper will have a big impact in the field. While this has its own merit, the biggest problem with that is there are just as many papers in those publications that have low citations as there are in any other journal.

While open access journals have begun a fight against this thinking, and journals such as PLoS One review potential articles without perceiving impact, I feel that there is a whole lot more that can be done to truly weigh impact factor. What’s more is that I want to apply this to open notebook science to (hopefully) show that open notebook articles can carry just as much weight, if not more, as a peer reviewed article in any journal.

And just how do I intend to do all that?

With website analytics!

Because of search engines, you no longer need to go to the library to find articles relevant to your field of study. You can just type something vague into a search engine bar, like tobacco seeds d2o – which has the top site as this notebook – and you can find all matter of information. Google Scholar allows you to narrow the search to just scholarly articles, and hopefully one day lab notebooks can be included in that search algorithm.

It is because of the ability to search, that people decided they needed some way to measure visitors to their sites. Someone created/developed/invented web analytics to track all sorts of fancy information about a visitor’s use of a website. Just about everyone on the web does this, and I thought “Why not scientists?”

Not all website analytics software is the same but generally speaking they all have the same basic functions:

  • track hits – number of visitors to the site
  • record movement – where in your site visitors navigate, and how long they stay there
  • referrers – did a visitor click a link from another website to get to yours? Was it a search engine, RSS reader, colleague, etc?
  • search results – which search engine a visitor used, and what they searched to find your page

While most of this is used for marketing purposes, it is still relevant for science. Being able to judge how many people read your open notebook consistently, seems to be something that scientists would like to know. Seeing that people are able to access your data through search results is useful and can help scientists optimize their results to target specific people. And finding out how people are coming across your information from sites that aren’t search engines, seems like it could be the new impact factor.

Twitter and Facebook have revolutionized how data is shared. I have a network of followers and friends and if I post something useful in my blog I share it to Twitter/Facebook. My network finds and reads it and then shares that with their network. That to me, is just like traditional citations because the people I’m targeting with my notebook entries are my peers, they read it, offer feedback and comments, and then share it with their peers. The act of sharing is the modern citation.

Not only that, but my commenting plugin (Disqus) actually counts the number of social media interactions and links to my notebook as “Reactions.” This gives me a solid number to say my notebook has been referenced “so many” times. Sounds like a citation to me.

Of course, people may not weigh this data the same, but I’m hoping that by acquiring actual numbers to associate to my notebook to go along with real world success stories, people will begin to weigh these citations similarly. I mean if all the major businesses in the world can count hits and use that as their metric for popularity and impact, why can’t science?

RC3: Day 9

DDW3: Day 2

I almost didn’t post these pictures, because nothing had happened yet. Then I looked at the tap water sample of arabidopsis seeds.

What’s that there?

Oh a sprout! Let the experiment begin!

RC3: Day 8

We are nearing the end of this trial. It appears that every seed that will germinate has already, obviously with the exception of the 99% d2o sample. Those seeds are never expected to grow. I’ll monitor the growth of these and setup another trial tomorrow.

Does ONS interfere with traditional scientific infrastructure?

After the BenchFly collaboration (see Advertising in Open Notebooks), Alan Marnett (CEO of BenchFly) was asked, and then asked me, a very interesting question:

Last night we received a comment on the poll about Open Notebooks wondering about the legal aspects of actually keeping a transparent notebook – do you have any feedback on that? It’s something I’ve wondered about too, since posting to a public blog/notebook would seem like it could be called a “publication”, which some journals may say prevents you from “republishing” with them.

Alan’s question got me thinking about this and the current state of science infrastructure with regards to open notebook science:

  1. Does publishing of any sort in an open notebook prevent publication in a journal?
  2. How does open notebook science impact current patent laws?
  3. What do funding agencies think about open notebook science and open science in general?

Since I didn’t have any direct feedback, I decided to seek out the answers. I sent out inquiries to some of the top publishers: Nature, Science, Elsevier, PLoS One, and Cell Press. I asked Jean-Claude Bradley and Steve Koch for some of their experiences as well, and now I’ve finally got enough data to publish a (this) report.

Aside: While acquiring data, I debated the fact that while this topic isn’t really science it is still open notebook worthy information. I wondered if I should be publishing my results as I get them or just make one long post. Obviously I opted for the one post, and while I didn’t publish my results, I did speak openly about my results up to that point in forums and in conversation. There was a discussion in the Linked In group, Open Science Supporters, about open notebooking and someone asked this very question. So it was there that I publicly revealed what I was researching and the preliminary results without posting any real data to back it up. Now let’s get back to the action.

Before getting in contact with publishers I browsed each publisher’s site to preview their official policies. All had documentation that was pretty similar to what Alan mentioned above, that a submission will be disqualified if there is a prior publication somewhere else. To put this in their own words let me show you some of what I found:

  • From Science: We will not consider any paper or component of a paper that has been published or is under consideration for publication elsewhere. Distribution on the Internet may be considered prior publication and may compromise the originality of the paper or submission. Please contact the editors with questions regarding allowable postings under this policy.
  • From PLoS One: PLoS ONE does not accept for publication work that has already been published elsewhere. However, studies that replicate results that are already in the literature may be considered for publication in PLoS ONE, as the independent confirmation of results can often be valuable, as can the presentation of a new dataset (for example, a new clinical trial). But then I was pointed to some more information: Authors may present and discuss their findings ahead of publication: at medical or scientific conferences, on preprint servers, in public databases, and in blogs, wikis, tweets, and other informal communication channels. We recommend, however, that authors not contact the media or respond to such contact unless an article has been accepted for publication and an embargo date has been established.

So, can an open notebook considered a prior publication? To answer this I wrote a generic inquiry to the publishers mentioned above with hopes of receiving a response that either answers my question or is a question that would allow me to clarify further, again to get an answer to my question:

My name is Anthony Salvagno. I’m an open notebook scientist which means that I publish my research data real time as experiments are carried out on a personal website (research.iheartanthony.com). I’ve been reading the submission guidelines for perspective authors and I’m wondering how this philosophy fits in with *publication-name* publication rights. If I publish my data, methods, etc before hand and submit to *publication-name* for peer review, is that a conflict of interest? Would I be disqualified from submission because of this? I’m very eager to hear an answer. Thanks for you time.

Of all the publishers, Science and Cell Press didn’t respond. Most replied via email, but Elsevier didn’t contact me either so I called them and spoke to someone from customer support in person. In summary, everyone (who I spoke with) said that as long as there is no formal write-up of my research (basically a paper) then data/results/commentary in an open notebook won’t be regarded as prior publishing. And in their own words…

  • Nicholas Ellinwood (publications assistant at PLoS One): Thank you for your email.  As long as your research was not formally published in another journal,  you can submit to PLoS ONE Please note that this is the policy of PLoS ONE specifically, and I cannot speak on behalf of the other PLoS journalsAlso, I would like to emphasize that research cannot be formally published in any capacity (not just science journals) before it is submitted to PLoS ONE.  PLoS does try to support open science, but we also commit to only publishing primary scientific research. It should be noted that we had a bit of an exchange and I only used the relevant information from those discussions here.
  • Dr. Maxine Clarke (Nature):Thank you for your message. The Nature journals support open notebook science. We encourage scientists to share and discuss data, methods and so on in advance of submission to one of our journals (a journal with “Nature” in the title) whether by your personal blog, wiki etc, by conference presentations or by posting on a recognized preprint server. What we ask potential and submitted authors not to do is to discuss their results with the media before publication. More details are here, but the above encapsulates the policy: http://www.nature.com/authors/policies/confidentiality.html http://www.nature.com/authors/policies/embargo.html Nature Publishing Group also publishes other journals, usually on behalf of scientific societies. These are likely to have their own individual policies depending on the society concerned, so if you want to submit to a journal published by us that isn’t a “Nature” journal, you’re advised to check out the website of the journal concerned, or send the editorial office a message via email (address is at each website) to enquire (sic).
  • Via Elsevier Customer Support: I contacted customer support and they said as long as there is no formal write up in the notebook then publishing in an open notebook should not interfere with the submission. I mentioned that my specific publication platform is a blog and he said that that shouldn’t be a problem as long as there is no formal write up. This is a general policy associated with Elsevier publications, and may be subject to minor discrepancies with regards to individual journals.

Best of all, there is even some precedent. Jean-Claude Bradley mentioned that his group published in the Journal of Visualized Experiments (JoVE) with research from their open notebook. He documented the experience here. At the time JoVE was an open access publication, but they have since switched to a more traditional publishing format (subscriptions). With that said, I have a hard time believing they would switch a core policy just because they went from open to closed but I have not researched that at all, so don’t take my word for it.

Jean-Claude also added that “we wrote the papers on an open wiki so that it became
formally a pre-print and was subject to publisher’s pre-print policies. Some publishers like ACS will not accept this. Now if we had not written the paper on a public wiki that might have been accepted even by ACS – and we could still have cited the lab notebook pages. We have not tested this though.”

So from the looks of it, publication is still possible but what about filing for a patent. For this I can only relay experience based on what we’ve seen as a lab.

Currently we have a patent application for our Shotgun DNA Mapping technique but it is for US only. Let’s let Steve Koch tell the story:

ONS immediately negates possibility for international patent protection. However, US patent protection is still possible within one year of publication (even speaking at an open seminar is pretty much publication). For SDM, we are only pursuing US protection. Once filing the disclosure with the USPTO, priority is established and further publication of the same information is fine. However, further innovations are subject to the same rules above. So, I (we) have made the decision to forgo international patent rights, and to seek US patent rights if we feel appropriate. UNM STC is fine with this. Other universities, and certainly many companies would not be fine with this.

So there you have it, international patents are currently incompatible with open notebook science, but US patent law allows for the filing of patent protection.

And how about funding? Do funding agencies frown upon open notebook science and open data?

Actually no! In all the grants that we’ve submitted (and it would be nice to hear about experiences from others) open notebook science and open science are actually the most positively received aspects of our submissions. I spoke with Steve about this and he mentioned that there are some grants that specifically don’t allow any kind of publication at all. In response, I added if that was the case we probably wouldn’t apply for those grants in general and I assume most scientists wouldn’t. After all, traditionally speaking career building is all about publications. If you can’t do that then you have different objectives anyway.

With the emergence of search and various web 2.0 tools, it seems much of the internet is shifting towards an open access approach (blogs, Youtube/Vimeo, Google Docs, etc). Science is following suit in many regards but open notebook science is still very new and very intimidating for most. Everyone sees the benefits of openly sharing and having access to protocols, methods, and results, but many are afraid to contribute because of possible conflicts with traditional scientific infrastructure (funding, patents, publications).

As more and more people follow an open notebook approach, the current system will need to change to meet the demands of the new user base. This will happen slowly, but I hope that I’ve been able to provide enough information, based on the experiences of our lab and those that I mentioned here, to be able to demonstrate that even right now ONS doesn’t interfere too much. Like I said, eventually the walls will come down.

Maybe one day primary research will come directly from the labs instead of peer-review. Following an experiment will be as easy as following your favorite blog (RSS). Scientists will reference their open notebook instead of their publication names. Collaborations will be easy and seamless. Someone has to take those first steps, and I don’t mind being the person to do it. Will you join me?

DDW3: Day 1

Today is much better from a setup standpoint. The seeds have soaked adequately and have sunk to the bottom. I tilted the cuvettes for a little so that all the seeds would be as close to the same focal plane as possible (at the front of the cuvette). Pictures taken indicate this.