Showing posts with label open science. Show all posts
Showing posts with label open science. Show all posts
Wednesday, April 4, 2012
PLOTS (publiclaboratory.org) is coming to #DavisCA
Saturday, February 4, 2012
Next up for Science in Congress: HR3433 - the Grant Reform and Transparency Act
Just got pointed to this by Mark Martin. There is a new bill making its way through congress - HR 3433 - the Grant Reform and New Transparency Act of 2011. It has a subtitle apparently of "To amend title 31, United States Code, to provide transparency and require certain standards in the award of Federal grants, and for other purposes."
The full text of the bill and other information is available here.
I personally don't know much about this bill but found some discussion of it here:
- House Committee Passes Bill Requiring Disclosure of Peer Reviewers
- GRANT Act Would Require Publication of All Research Grant ...
- Federal Relations - Duke Federal Relations: DC Digest
- FABBS :: GRANT Act Draws Concerns of Scientists and Policymakers
- Peer Review Under Attack in New House Bill « Physics FrontlineNew grant transparency act introduced | AAAS MemberCentral
- Letter to House Leadership Seeks Changes in “Troubling Provisions ...
- GRANT Transparency good, DUNS numbers bad - Sunlight ...
I have not formed an opinion of the act but thought I would share the information since it does not seem to be getting much attention but seems like it could have impact. I note - one group that I respect deeply supports the act: the Sunlight Foundation which involves people like Ester Dyson and Lawrence Lessig. Any opinions or insight on the bill would be welcome.
Stop deifying "peer review" of journal publications:
Peer review. It is a critical part of scientific research and scientific progress. Without it, science as a field might look like Fox News Stories or postings on Jenny McCarthy's web site, where ideas people have are given gravitas regardless of how ludicrous they are. But somehow, many in the public and press, and many many scientists alas, have deep misconceptions about peer review.
The most recent example of such misconceptions involves the arsenic life saga. If you are not familiar with this story - here is a summary (for some fine scale details on the early parts of the story see Carl Zimmer's post here).
In November 2010 NASA announced that in a few days they would hold a press conference discussing a major finding about life in the universe. On December 2, 2010, they held their press conference and discussed a paper that was in press in Science from multiple NASA funded authors including Felisa Wolfe-Simon. The paper was of interest because it claimed to have shown that a bacterium was able to replace phosphate in its macromolecules, including its DNA, with arsenic. The press conference made claims that were very grandiose, like that textbooks would have to be rewritten, and the study of life on Earth and elsewhere would have to be completely rethought.
After a few days of mostly very glowing press reports, a few critiques began to emerge including in particular one from Rosie Redfield, a microbiologist at the University of British Columbia. The critiques then snowballed and snowballed and the general consensus of comments appeared to be that the paper had fundamental flaws. Some of the critiques got way too personal in my opinion and I begged everyone to focus on the science not personal critiques. This seemed to work a little bit and we could focus on the science, which still seemed to be dubious. And many, including myself, expressed the opinion that the claims made by the authors in the paper and by the authors and NASA in the press conference and in comments to the press, were misleading at best.
Now critiques about new findings are not unusual. We will get back to that in a minute. But what was astonishing to me and many others, was how NASA and the authors responded. They said things like:
And so the saga continued. Rosie Redfield began to do experiments to test some of the work reported in the paper. Many critiques of the original paper were published. The actual paper finally came out. And many went about their daily lives (I keep thinking of the Lord of the Rings whisper "History became legend. Legend became myth. And for two and a half thousand years, the ring passed out of all knowledge." Alas, the arsenic story did not go away.
And now skipping over about a year. The arsenic story came back into our consciousness thanks to the continued work of Rosie Redfield. And amazingly and sadly, Wolfe-Simon's response to Rosie's work included a claim that they never said that arsenic was incorporate into the bacterium's DNA. (I have posted a detailed refutation of this new "not in DNA" comment here).
But that is not what I am writing about here. What is also sad to me are the continued statements by the paper's authors that they will not discuss any critiques or work of others unless they are published in a peer reviewed article.
For example, see Elizabeth Pannisi's article in Science:
Though much has been written about peer review already (e.g., see Peer review discussion at Nature as one example), I would like to add my two cents now - focusing on the exalted status some give to peer reviewed journal articles. I have three main concerns with this attitude which can be summarized as follows
Regarding #1 "Peer review is not magic.".
What I mean by this is that peer review is not something that one can just ask for and "poof" it happens. Peer review of articles (or any other type of peer review for that matter) frequently does not work as sold - work that is poor can get published and work that is sound can get rejected. While it may pain scientists to say this (and brings up fears of FoxNews abusing findings) it is alas true. It is not surprising however given the way articles get reviewed.
In summary this is how the process works. People write a paper. They then submit it to a journal. An editor or editors at the journal decide whether or not to even have it reviewed. If they decide "no" the paper is "sent back" to the authors and then they are free to send it somewhere else. If they decide "yes" to review it, the editors then ask a small number of "peers" to review the article (the number usually ranges from 2-3 in my field). Peers then send in comments to the editor(s) and the editor(s) then make a "decision" and relay that decision to the authors. They may say the paper is rejected. Or they may say it is accepted. Or they may say "If you address the comments of the reviewers, we would consider accepting it". And then the authors can make some revisions and send it back to the editors. Then it is reviewed again (sometimes just by the editors, sometimes by "peers"). And it may be accepted or rejected or sent back for more revisions. And so on.
In many cases, the review by peers is insightful, detailed, useful and in the best interests of scientific progress. But in many cases the review is flawed. People miss mistakes. People are busy and skim over parts of the paper. People have grudges and hide behind anonymity. People can be overly nice in review if the paper is from friends. People may not understand some of the details but may not let the editors know. Plus - the editors are not completely objective in most cases either. Editors want "high profile" papers in many cases. They want novelty. They want attention. This may lead them to ignore possible flaws in a paper in exchange for the promise that it holds. Editors also have friends and enemies. And so on. In the end, the "peer review" that is being exalted by many is at best the potentially biased opinion of a couple of people. At worst, it is a steaming pile of ... Or, in other words, peer review is imperfect. Now, I am not saying it is completely useless, as peer review of journal articles can be very helpful in many ways. But it should be put in its rightful place.
Regarding #2: "Peer review is not binary"
The thumbs up / thumbs down style of peer review of many journal articles is a major flaw. Sure - it would be nice if we could apply such a binary metric. And this would make discussing science with the press and the public so much easier "No ma'am, I am sorry but that claim did not pass peer review so I cannot discuss it" "Yes sir, they proved that because their work cleared peer review." But in reality, papers are not "good" or "bad". They have good parts and bad parts and everything in between. Peer review or articles should be viewed as a sliding scale and not a "yes" vs. "no."
---------------------------
UPDATE 1: Some discussions of this post
UPDATE 3: some twitter comments
The most recent example of such misconceptions involves the arsenic life saga. If you are not familiar with this story - here is a summary (for some fine scale details on the early parts of the story see Carl Zimmer's post here).
In November 2010 NASA announced that in a few days they would hold a press conference discussing a major finding about life in the universe. On December 2, 2010, they held their press conference and discussed a paper that was in press in Science from multiple NASA funded authors including Felisa Wolfe-Simon. The paper was of interest because it claimed to have shown that a bacterium was able to replace phosphate in its macromolecules, including its DNA, with arsenic. The press conference made claims that were very grandiose, like that textbooks would have to be rewritten, and the study of life on Earth and elsewhere would have to be completely rethought.
After a few days of mostly very glowing press reports, a few critiques began to emerge including in particular one from Rosie Redfield, a microbiologist at the University of British Columbia. The critiques then snowballed and snowballed and the general consensus of comments appeared to be that the paper had fundamental flaws. Some of the critiques got way too personal in my opinion and I begged everyone to focus on the science not personal critiques. This seemed to work a little bit and we could focus on the science, which still seemed to be dubious. And many, including myself, expressed the opinion that the claims made by the authors in the paper and by the authors and NASA in the press conference and in comments to the press, were misleading at best.
Now critiques about new findings are not unusual. We will get back to that in a minute. But what was astonishing to me and many others, was how NASA and the authors responded. They said things like:
... we hope to see this work published in a peer-reviewed journal, as this is how science best proceeds.and
It is one thing for scientists to “argue” collegially in the public media about diverse details of established notions, their own opinions, policy matters related to health/environment/science.
But when the scientists involved in a research finding published in scientific journal use the media to debate the questions or comments of others, they have crossed a sacred boundary [via Carl Zimmer]and the kicker for me was a letter Zimmer posted
Mr. Zimmer,
I am aware that Dr. Ronald Oremland has replied to your inquiry. I am in full and complete agreement with Dr. Oremland’s position (and the content of his statements) and suggest that you honor the way scientific work must be conducted.
Any discourse will have to be peer-reviewed in the same manner as our paper was, and go through a vetting process so that all discussion is properly moderated. You can see many examples in the journals Science and Nature, the former being where our paper was published. This is a common practice not new to the scientific community. The items you are presenting do not represent the proper way to engage in a scientific discourse and we will not respond in this manner.
Regards,This was amazing since, well, they were the ones who held the overhyped press conference. And then I (and others) found it appalling that they in essence would not response to critiques because they were not "peer reviewed." I told Zimmer
Felisa
Whether they were right or not in their claims, they are now hypocritical if they say that the only response should be in the scientific literature.Zimmer had a strong defense of scientists "discussing" the paper:
Of course, as I and others have reported, the authors of the new paper claim that all this is entirely inappropriate. They say this conversation should all be limited to peer-reviewed journals. I don’t agree. These were all on-the-record comments from experts who read the paper, which I solicited for a news article. So they’re legit in every sense of the word. Who knows–they might even help inform peer-reviewed science that comes out later on.(I note - yes I am quoting a lot from Zimmer's articles on the matter and there are dozens if not hundreds of others - apologies to those out there who I am not referencing - will try to dig in and add other references later if possible).
And so the saga continued. Rosie Redfield began to do experiments to test some of the work reported in the paper. Many critiques of the original paper were published. The actual paper finally came out. And many went about their daily lives (I keep thinking of the Lord of the Rings whisper "History became legend. Legend became myth. And for two and a half thousand years, the ring passed out of all knowledge." Alas, the arsenic story did not go away.
And now skipping over about a year. The arsenic story came back into our consciousness thanks to the continued work of Rosie Redfield. And amazingly and sadly, Wolfe-Simon's response to Rosie's work included a claim that they never said that arsenic was incorporate into the bacterium's DNA. (I have posted a detailed refutation of this new "not in DNA" comment here).
But that is not what I am writing about here. What is also sad to me are the continued statements by the paper's authors that they will not discuss any critiques or work of others unless they are published in a peer reviewed article.
For example, see Elizabeth Pannisi's article in Science:
But Wolfe-Simon and her colleagues say the work on arsenic-based life is just beginning. They told ScienceInsider that they will not comment on the details of Redfield's work until it has been peer reviewed and published.So - enough of an introduction. What is it I wanted to write about peer review? What I want to discuss here is that the deification of a particular kind of journal peer review by the arsenic-life authors is alas not unique. There are many who seem to have similar feelings (e.g., see this defense of the Wolfe-Simon position). I believe this attitude towards peer review is bad for science. Fortunately, many others agree (e.g., see this rebuttal of the defense mentioned above) and there is a growing trend to expand the concepts of what peer review is and what it means (see for example, David Dobbs great post about peer review and open science from yesterday).
Though much has been written about peer review already (e.g., see Peer review discussion at Nature as one example), I would like to add my two cents now - focusing on the exalted status some give to peer reviewed journal articles. I have three main concerns with this attitude which can be summarized as follows
- Peer review is not magic
- Peer review is not binary
- Peer review is not static.
Regarding #1 "Peer review is not magic.".
What I mean by this is that peer review is not something that one can just ask for and "poof" it happens. Peer review of articles (or any other type of peer review for that matter) frequently does not work as sold - work that is poor can get published and work that is sound can get rejected. While it may pain scientists to say this (and brings up fears of FoxNews abusing findings) it is alas true. It is not surprising however given the way articles get reviewed.
In summary this is how the process works. People write a paper. They then submit it to a journal. An editor or editors at the journal decide whether or not to even have it reviewed. If they decide "no" the paper is "sent back" to the authors and then they are free to send it somewhere else. If they decide "yes" to review it, the editors then ask a small number of "peers" to review the article (the number usually ranges from 2-3 in my field). Peers then send in comments to the editor(s) and the editor(s) then make a "decision" and relay that decision to the authors. They may say the paper is rejected. Or they may say it is accepted. Or they may say "If you address the comments of the reviewers, we would consider accepting it". And then the authors can make some revisions and send it back to the editors. Then it is reviewed again (sometimes just by the editors, sometimes by "peers"). And it may be accepted or rejected or sent back for more revisions. And so on.
In many cases, the review by peers is insightful, detailed, useful and in the best interests of scientific progress. But in many cases the review is flawed. People miss mistakes. People are busy and skim over parts of the paper. People have grudges and hide behind anonymity. People can be overly nice in review if the paper is from friends. People may not understand some of the details but may not let the editors know. Plus - the editors are not completely objective in most cases either. Editors want "high profile" papers in many cases. They want novelty. They want attention. This may lead them to ignore possible flaws in a paper in exchange for the promise that it holds. Editors also have friends and enemies. And so on. In the end, the "peer review" that is being exalted by many is at best the potentially biased opinion of a couple of people. At worst, it is a steaming pile of ... Or, in other words, peer review is imperfect. Now, I am not saying it is completely useless, as peer review of journal articles can be very helpful in many ways. But it should be put in its rightful place.
Regarding #2: "Peer review is not binary"
The thumbs up / thumbs down style of peer review of many journal articles is a major flaw. Sure - it would be nice if we could apply such a binary metric. And this would make discussing science with the press and the public so much easier "No ma'am, I am sorry but that claim did not pass peer review so I cannot discuss it" "Yes sir, they proved that because their work cleared peer review." But in reality, papers are not "good" or "bad". They have good parts and bad parts and everything in between. Peer review or articles should be viewed as a sliding scale and not a "yes" vs. "no."
Regarding #3: "Peer review is not static"
This is perhaps the most important issue to me in peer review of scientific work. Peer review of journal articles (as envisioned by many) is a one time event. Once you get the thumbs up - you are through the gate and all is good forever more. But that is just inane. Peer review should be - and in fact with most scientists is - continuous. It should happen before, during and after the "peer review" that happens for a publication. Peer review happens at conferences - in hallways - in lab meetings - on the phone - on skype - on twitter - at arXiv - in the shower - in classes - in letters - and so on. Scientific findings need to be constantly evaluated - tested - reworked - critiqued - written about - discussed - blogged - tweeted - taught - made into art - presented to the public - turned inside out - and so on.
Summary:
In the end - what people should understand about peer review is that though it is not perfect, it can be done well. And the key to doing it well is to view it as a continuous, nuanced activity and not a binary, one time event.
---------------------------
UPDATE 1: Some discussions of this post
- On my Google+
- On Dennis McDonald's Google+
- Universal open review? from Rich Jorgensen
- Peer Review Meets DIY: Publishing a Student Science Journal
- Tiptoeing Toward the Tipping Point | Peer to Peer Review
- Crackpot 'Theory of Everything' Reveals Dark Side of Peer Review
- F1000 Research: post-publication peer review and data sharing ...
- Distributed Peer Review | Blogging Pedagogy
- Publishers do not provide peer-review. We do. « Sauropod Vertebra ...
- Time for a new type of peer review?
1. Peer review is not magic2. Peer review is not binary3. Peer review is not static.phylogenomics.blogspot.com/2012/02/stop-d… by @phylogenomics
— figshare (@figshare) February 4, 2012
It's not magic, binary or static. MT @phylogenomics: Stop deifying "peer review" of journal publications: goo.gl/fb/3kIEN
— Stephen Curry (@Stephen_Curry) February 4, 2012
Some home truths about what peer review is (and isn't) - excellent stuff from @phylogenomics goo.gl/fb/3kIEN Via @Stephen_Curry
— Dr Aust (@Dr_Aust_PhD) February 4, 2012
All interested in explaining science, read @phylogenomics's blog post on peer review phylogenomics.blogspot.com/2012/02/stop-d… -- these are important points.
— Chris Gunter (@girlscientist) February 4, 2012
Friday, February 3, 2012
Interesting new metagenomics paper w/ one big big big caveat - critical software not available "
Very very strange. There is an interesting new metagenomics paper that has come out in Science this week. It is titled "Untangling Genomes from Metagenomes: Revealing an Uncultured Class of Marine Euryarchaeota" and it is from the Armbrust lab at U. Washington.
One of the main points of this paper is that the lab has developed software that apparently can help assemble the complete genomes of organisms that are present in low abundance in a metagenomic sample. At some point I will comment on the science in the paper, (which seems very interesting) though as the paper in non Open Access I feel uncomfortable doing so since many of the readers of this blog will not be able to read it.
But something else relating to this paper is worth noting and it is disturbing to me. In a Nature News story on the paper by Virginia Gewin there is some detail about the computational method used in the paper:
One of the main points of this paper is that the lab has developed software that apparently can help assemble the complete genomes of organisms that are present in low abundance in a metagenomic sample. At some point I will comment on the science in the paper, (which seems very interesting) though as the paper in non Open Access I feel uncomfortable doing so since many of the readers of this blog will not be able to read it.
But something else relating to this paper is worth noting and it is disturbing to me. In a Nature News story on the paper by Virginia Gewin there is some detail about the computational method used in the paper:
"He developed a computational method to break the stitched metagenome into chunks that could be separated into different types of organisms. He was then able to assemble the complete genome of Euryarchaeota, even though it was rare within the sample. He plans to release the software over the next six months."What? It is imperative that software that is so critical to a publication be released in association with the paper. It is really unacceptable for the authors to say "we developed a novel computational method" and then to say "we will make it available in six months". I am hoping the authors change their mind on this but I find it disturbing that Science would allow publication of a paper highlighting a new method and then not have the method be available. If the methods and results in a paper are not usable how can one test/reproduce the work?
Sunday, January 29, 2012
Dear #AAAS, I am NOT embargoing my own talk & I plan to record it and post afterwards #embargowatch
Just got another email from AAAS regarding their big meeting in February in Vancouver where I am scheduled to talk:
----------------------------------------The thing is - I did not agree to "Embargo" my talk and as I wrote about before, I do not even know what that means. I figured, in the interest of being "open" about my feelings about this, I should write to AAAS to let them know I was not going to embargo my own talk, and I plan to record my talk and post it afterwards:
This request for materials is from the AAAS media relations team and is separate from any you may receive from your symposium organizer or the AAAS Annual Meeting office.
----------------------------------------
Dear AAAS Annual Meeting Participant:
Thanks to all of you who uploaded materials to the AAAS Virtual Newsroom by Jan. 16. For those of you who have not submitted materials or want to submit additional materials, you may do so right up through the meeting. The materials will be available online to reporters, although we can no longer guarantee that we'll be able to copy new
submissions at our expense for placement in the on-site library of speaker materials. We will try to include materials received in the next several days in our copy order, however.
You also can make printed copies (10-15 copies) yourself and ship them to Vancouver so that we can place them in the on-site papers library for reporters. Ideally, press materials should be on-site prior to your presentation. Please see below for appended mailing instructions.
Speakers and organizers can submit materials by going to:
http://www.eurekalert.org/aaasnewsroom/mcm/speakers
Your individual username and password for the site:
Please provide the following:
-- A one-paragraph biographical sketch (not a C.V.)
-- A short lay-language summary of your talk, beyond the abstract.
-- The text of your talk, if available, or a related (ideally recent) technical paper, either as a Word file or a PDF. PowerPoint presentations are acceptable, but a full text will better serve reporters' needs.
-- Any additional supporting materials, including multimedia files such as JPEG or TIFF photos in high resolution (300 dpi) and/or digitized video clips.
IMPORTANT: Please note that all AAAS meeting presentations are strictly embargoed and your speaker materials should not be released publicly until the time of your presentation.
If you upload your materials by 16 January, we will copy them at our expense for placement in the on-site library of speaker materials, available only to newsroom registrants.
Please notify your institution's press office of your AAAS Annual Meeting presentation as soon as possible. Your press office can help you submit speaker materials to us and can begin to generate media interest.
....
To whom it may concernWill report back if I get a reply ... and maybe I can get Ivan Oransky to help make sense out of what a talk embargo means.
I am scheduled to speak at the AAAS meeting and I am writing this in regard to the email attached below. I do not support the notion of an Embargo for my talk and I am unwilling to participate in the embargo. I plan to post information about my talk to the web and to my blog and am writing to specifically let you know I fundamentally do not support the embargo nor did I agree to it when I agreed to give a talk at AAAS.
I also plan to record my own talk and to post the recording and the slides to various websites. I am not sure if AAAS has a policy about that but wanted to let you know of my plans in the interest of not having any surprises.
Sincerely
Jonathan Eisen
Friday, January 13, 2012
Request for input - are there any rules regarding posting text of one's own NSF (or other) grant proposals?
In response to a series of posts from Karen James (who is a biologist now in Maine and is director of the HMS Beagle Project) on Twitter, I am posting here to ask for input from the crowd. On Twitter, Karen has been discussing her putting together an NSF proposal and was then celebrating a few days ago when it was done.
I have posted some of the twitter conversation below. But to get directly to the point the question I have for everyone here is - are there any rules at the National Science Foundation that would prevent one from sharing with others a grant proposal that one has submitted? Are there any rules against this at any agency? I think there are none but apparently some are telling Karen otherwise.
Any information on this would be useful. Some of the twitter conversation is below:
kejames
So, @phylogenomics and others, with whom is it appropriate to share a submitted NSF proposal? Anyone? No one? Something in between?
1/12/12 6:15 PM
phylogenomics
@kejames what do you mean by "appropriate"
1/12/12 6:15 PM
johnhawks
@kejames If I were you, I'd share the whole thing in public and make the reviews public as well. But I'm a minority view.
1/12/12 6:16 PM
kejames
@johnhawks Thanks, I've thought of that, actually. It is a federal agency after all. I'd need to redact confidential budget info, though.
1/12/12 6:18 PM
johnhawks
@kejames Yes, and possibly key personnel. My attitude is the success rate is so low, it can't hurt and might draw visibility pre-review.
1/12/12 6:19 PM
kejames
@phylogenomics Best practice. My instinct is to share it with colleagues, collaborators and associates I think might be interested in it.
1/12/12 6:16 PM
phylogenomics
@kejames what is the potential reason to not share?
1/12/12 6:17 PM
kejames
@phylogenomics That's what I'm asking. Is there any rule or custom that prohibits sharing it far and wide?
1/12/12 6:19 PM
phylogenomics
@kejames none that I know of - only reason not to is if you are worried about people "stealing" your ideas
1/12/12 6:19 PM
kejames
@phylogenomics Not worried about that in the slightest. If anything sharing it widely establishes it as "my" idea. Thanks.
1/12/12 6:20 PM
kejames
@johnhawks Or I could ask the key personnel if they're okay w/ it. I think it would be nice to include them if they want to be included.
1/12/12 6:21 PM
phylogenomics
@kejames I think Rosie Redfield posts hers on her blog http://t.co/g73Xb2Yz
1/12/12 6:25 PM
kejames
@phylogenomics Thanks for that. I notice she just posts the project description itself, none of the other "stuff"and doesn't list names.
1/12/12 6:28 PM
DoctorZen
@kejames NSF proposals are your choice who to share with. Probably not best to post publicly, though.
1/12/12 6:25 PM
kejames
@DoctorZen Why not? As @phylogenomics notes, @RosieRedfield posts her grant proposals on her lab's website: http://t.co/peWEmnvs
1/12/12 6:29 PM
kejames
Anyone else besides @phylogenomics @doctorzen @johnhawks want to weigh in on how broadly I should share my just-submitted NSF proposal? 1/2
1/12/12 6:31 PM
kejames
@rdmpage @RosieRedfield @DoctorZen @johnhawks @phylogenomics @kzelnio So I asked the collaborators on the proposal. One replied… 1/2
1/13/12 4:50 AM
kejames
@rdmpage @RosieRedfield @DoctorZen @johnhawks @phylogenomics @kzelnio 2/2…"Sharing not wise! Could disqualify proposal."
1/13/12 4:51 AM
rdmpage
.@kejames @rosieredfield @doctorzen @johnhawks @phylogenomics @kzelnio In other words fear of what grant agency will do trumps being open :(
1/13/12 5:10 AM
DoctorZen
@rdmpage I support being open; not sure every step always needs to be public. @kejames @rosieredfield @johnhawks @phylogenomics @kzelnio
1/13/12 5:18 AM
rdmpage
.@DoctorZen @kejames @rosieredfield @johnhawks @phylogenomics @kzelnio I agree, it's not that it HAS to open, but that it COULD be
1/13/12 5:27 AM
phylogenomics
@kejames @rdmpage @RosieRedfield @DoctorZen @johnhawks @kzelnio WTF? As far as I know there are NO NSF issues w/ sharing a proposal
1/13/12 7:00 AM
phylogenomics
@kejames @rdmpage @RosieRedfield @DoctorZen @johnhawks @kzelnio Yes, need to discuss w/ collabs & get permission but not angst any rules
1/13/12 7:02 AM
phylogenomics
@rdmpage @DoctorZen @kejames @rosieredfield @johnhawks @kzelnio agree w/ Rod - issue was whether it could be posted, not if it had to be
1/13/12 7:04 AM
kejames
@phylogenomics I'm following up w/ him to find out what he meant He's a seasoned NSF grantee and reviewer. Have also contacted NSF directly.
1/13/12 7:08 AM
I have posted some of the twitter conversation below. But to get directly to the point the question I have for everyone here is - are there any rules at the National Science Foundation that would prevent one from sharing with others a grant proposal that one has submitted? Are there any rules against this at any agency? I think there are none but apparently some are telling Karen otherwise.
Any information on this would be useful. Some of the twitter conversation is below:
kejames
So, @phylogenomics and others, with whom is it appropriate to share a submitted NSF proposal? Anyone? No one? Something in between?
1/12/12 6:15 PM
phylogenomics
@kejames what do you mean by "appropriate"
1/12/12 6:15 PM
johnhawks
@kejames If I were you, I'd share the whole thing in public and make the reviews public as well. But I'm a minority view.
1/12/12 6:16 PM
kejames
@johnhawks Thanks, I've thought of that, actually. It is a federal agency after all. I'd need to redact confidential budget info, though.
1/12/12 6:18 PM
johnhawks
@kejames Yes, and possibly key personnel. My attitude is the success rate is so low, it can't hurt and might draw visibility pre-review.
1/12/12 6:19 PM
kejames
@phylogenomics Best practice. My instinct is to share it with colleagues, collaborators and associates I think might be interested in it.
1/12/12 6:16 PM
phylogenomics
@kejames what is the potential reason to not share?
1/12/12 6:17 PM
kejames
@phylogenomics That's what I'm asking. Is there any rule or custom that prohibits sharing it far and wide?
1/12/12 6:19 PM
phylogenomics
@kejames none that I know of - only reason not to is if you are worried about people "stealing" your ideas
1/12/12 6:19 PM
kejames
@phylogenomics Not worried about that in the slightest. If anything sharing it widely establishes it as "my" idea. Thanks.
1/12/12 6:20 PM
kejames
@johnhawks Or I could ask the key personnel if they're okay w/ it. I think it would be nice to include them if they want to be included.
1/12/12 6:21 PM
phylogenomics
@kejames I think Rosie Redfield posts hers on her blog http://t.co/g73Xb2Yz
1/12/12 6:25 PM
kejames
@phylogenomics Thanks for that. I notice she just posts the project description itself, none of the other "stuff"and doesn't list names.
1/12/12 6:28 PM
DoctorZen
@kejames NSF proposals are your choice who to share with. Probably not best to post publicly, though.
1/12/12 6:25 PM
kejames
@DoctorZen Why not? As @phylogenomics notes, @RosieRedfield posts her grant proposals on her lab's website: http://t.co/peWEmnvs
1/12/12 6:29 PM
kejames
Anyone else besides @phylogenomics @doctorzen @johnhawks want to weigh in on how broadly I should share my just-submitted NSF proposal? 1/2
1/12/12 6:31 PM
kejames
@rdmpage @RosieRedfield @DoctorZen @johnhawks @phylogenomics @kzelnio So I asked the collaborators on the proposal. One replied… 1/2
1/13/12 4:50 AM
kejames
@rdmpage @RosieRedfield @DoctorZen @johnhawks @phylogenomics @kzelnio 2/2…"Sharing not wise! Could disqualify proposal."
1/13/12 4:51 AM
rdmpage
.@kejames @rosieredfield @doctorzen @johnhawks @phylogenomics @kzelnio In other words fear of what grant agency will do trumps being open :(
1/13/12 5:10 AM
DoctorZen
@rdmpage I support being open; not sure every step always needs to be public. @kejames @rosieredfield @johnhawks @phylogenomics @kzelnio
1/13/12 5:18 AM
rdmpage
.@DoctorZen @kejames @rosieredfield @johnhawks @phylogenomics @kzelnio I agree, it's not that it HAS to open, but that it COULD be
1/13/12 5:27 AM
phylogenomics
@kejames @rdmpage @RosieRedfield @DoctorZen @johnhawks @kzelnio WTF? As far as I know there are NO NSF issues w/ sharing a proposal
1/13/12 7:00 AM
phylogenomics
@kejames @rdmpage @RosieRedfield @DoctorZen @johnhawks @kzelnio Yes, need to discuss w/ collabs & get permission but not angst any rules
1/13/12 7:02 AM
phylogenomics
@rdmpage @DoctorZen @kejames @rosieredfield @johnhawks @kzelnio agree w/ Rod - issue was whether it could be posted, not if it had to be
1/13/12 7:04 AM
kejames
@phylogenomics I'm following up w/ him to find out what he meant He's a seasoned NSF grantee and reviewer. Have also contacted NSF directly.
1/13/12 7:08 AM
Sunday, January 1, 2012
Draft post cleanup #3: The Open Knowledge Foundation
Yet another post in my "draft blog post cleanup" series. Here is #3 from just a few weeks ago:
Interesting article in PLoS Biology: PLoS Biology: The Open Knowledge Foundation: Open Data Means Better Science. It discusses many issues in open science especially as they relate to open data.
Some links from this paper are worth checking out
Interesting article in PLoS Biology: PLoS Biology: The Open Knowledge Foundation: Open Data Means Better Science. It discusses many issues in open science especially as they relate to open data.
Some links from this paper are worth checking out
- http://www.opendefinition.org/okd/
- http://science.okfn.org/About/
- http://www.earlham.edu/~peters/fos/bethesda.htm
- http://sciencecommons.org/projects/publishing/open-access-data-protocol/
- http://www.pantonprinciples.org/
- http://www.isitopendata.org/
This article reminds me that I keep meaning to push for the development of a "Datawatch" system much like the "RetractionWatch" systems of Ivan Oransky. I have discussed this with Ivan but we have not yet gotten around to doing it ...
Thursday, September 8, 2011
Garage science: why skyping in to a meeting at 2:30 AM can be, well, interesting #SOLO11
I got an email many months ago inviting me to participate in a panel discussion for a Science Online Meeting in London. The discussion was to focus on the NASA arsenic life story and how this related to open science issues. Now even though the one and only Kaitlin Thaney invited me, I alas said "it's going to be hard for me to get to London at that time" and eventually we settled on the idea of me skyping in for the panel.
And I sort of forgot about this for a bit.
And then Joerg Heber emailed me with some comments about the panel and there was an email discussion among the panelists about details. And in the midst of all of this Joerg pointed out that the panel was at 10:15 AM London time. All along I had thought it was going to be in the evening in London. 10:15 AM London time is 2:15 AM California time. And thus I started to freak out a little bit. How was I going to pull this off? I have two young kids. And my house is very open in design so there is nowhere to go where people can't hear you. I would wake up everyone if I talked on a panel from inside my house ... hmmm
Anyway - here are some twitter posts that will give you a feel for how things played out from a mechanical point of view:
silentypewriter Ed Gerstner
Getting psyched for panel discussion with @ivanoransky @rosieredfield @phylogenomics @joergheber at #solo11bit.ly/rtSwpY2 Sep
phylogenomics Jonathan Eisen
Perils of an open house design; am skyping in to #scio11 panel from home at 2:15 AM but have to do it from garage to avoid waking up kids
2 Sep
kristiholmes Kristi Holmes
@phylogenomics I gave an online talk last year @ ORCID participants meeting in almost the same manner (3am local time) #coffeeisessential2 Sep
phylogenomics Jonathan Eisen
@kristiholmes the cricket in the background will be fun
2 Sep
phylogenomics Jonathan Eisen
Perils of an open house design; am skyping in to #solo11 panel from home at 2:15 AM but have to do it from garage to avoid waking up kids
2 Sep
ryneches Russell Neches
@phylogenomics You have a hot tub now. You should phone it in from there, like a movie villain.
2 Sep
sjcockell Simon Cockell
Skyping in @phylogenomics for the panel session now...twitpic.com/6eswaa 2 Sep
lualnu10 Marisa Alonso Nuñez
3 present speakers and the 4th one on skype (it's 2:30 am for him) lol #solo11 2 Sep
LouWoodley Lou Woodley
"I don't think you want my video" - big star @phylogenomicsskype-ing in for panel discussion on linking with the literature #solo112 Sep
rmounce Ross Mounce
"I dont think you want my video" @phylogenomics LOL! (skyping in to #solo11 at 2am from the garage)
2 Sep
vibjpatel Vibhuti J. Patel
Truly online conferencing at #solo11 - Skyping in a panellist from California for the discussion on Linking with the Literature #arseniclife 2 Sep
@ryneches Russell Neches
@phylogenomics Right. VILLAIN. Get yourself a chunky cordless phone and some dudes in sunglasses who stand around looking menacing.
2 Sep via web
phylogenomics Jonathan Eisen
This is what skyping from your garage at 2:30 AM in California looks like for all you #solo11 folks post.ly/33cSm 2 Sep
kaythaney Kaitlin Thaney
Dedication :) RT @phylogenomics: This is what skyping from your garage at 2:30 AM in California looks like #solo11 post.ly/33cSm 2 Sep
That's right ... I skyped in from my garage. In my pajamas. And thus I refused the video connection. But other than being exhausted the rest of the day, I think things worked out well. I will post later with details of the panel and the SOLO meeting since there are some good reports out there. And I hope audio/video will be posted some time too.
Some links about the meeting:
- Changed by Science Online London 2011?
- Science Online London 2011 Day 1: Storify of #1 Panel: Linking with the Literature - The Arsenic Story #solo11
- Science Online London 2011 Day 1: Storify of the Keynote by Michael Nielsen #solo11
- Incentives: Definitely a case of rolling your own
- Science Online London Keynote, Michael Nielson on Open Sciecne
- SOLO11, day1, morning sessions.
- SOLO11: Science online London 2011 (#solo11) - day 2, panel 1 - "Dealing with Data"
Wednesday, September 7, 2011
My science communication hero/heroine of the month - Dr. Kiki @drkiki
Been working on revising my lab's web site and was looking for some videos of talks I have given online to post there. And I discovered/rediscovered this video of an interview I did for Dr. Kiki's Science Hour. Here it is:
NOTE - AT LEAST TEMPORARILY REMOVING THE VIDEO DUE TO MALWARE INFECTION OF TWIT.TV SITE
Now I know - this is over a year old. But I just watched the full video. Not so bad I think.
As many of you know, I like to talk. And talk. And talk. But I would like to say that as an interviewer, Dr. Kiki is pretty frigging awesome. Don't know how she does it. But I am going to post this video on the new lab page and point people to it if they want to know what my lab does and what I am interested in.
But enough about me. I want to thank Dr. Kiki for this great interview by saying a little bit about her. Or, well, her work in science communication.
As some of you may know, I listen to podcasts of TWIS - This Week in Science frequently on my bike rides to work. And I really recommend anyone/everyone out there give it a whirl. It is sort of like Science Friday but it is a bit edgier, a bit funnier, a bit goofier, and a bit sciencier (is that a word?) Dr. Kiki and Justin on it are great and it is so good that I frequently sit outside my building listening to the end of a show if I take the short ride to work which is less than an hour. So if you like Science - you really should check out the TWIS web site and find some way to listen such as what I do by subscribing to their podcasts at iTunes.
And I guess now I will be checking out "Dr. Kiki's Science Hour" more after rewatching this video. There are many many more shows at twit.tv/kiki. I have not checked out as many as TWIS shows but the ones I have watched are great.
And if you want to follow her more directly check out her Blog: The Bird's Brain, or her twitter feed (@drkiki) or her Google+ feed.
Very proud that she is a UC Davis alum ... and just want to say thanks to her for giving me a video I can share with others that says more about me and my lab than almost anything I have written.
NOTE - AT LEAST TEMPORARILY REMOVING THE VIDEO DUE TO MALWARE INFECTION OF TWIT.TV SITE
Now I know - this is over a year old. But I just watched the full video. Not so bad I think.
As many of you know, I like to talk. And talk. And talk. But I would like to say that as an interviewer, Dr. Kiki is pretty frigging awesome. Don't know how she does it. But I am going to post this video on the new lab page and point people to it if they want to know what my lab does and what I am interested in.
But enough about me. I want to thank Dr. Kiki for this great interview by saying a little bit about her. Or, well, her work in science communication.
And if you want to follow her more directly check out her Blog: The Bird's Brain, or her twitter feed (@drkiki) or her Google+ feed.
Very proud that she is a UC Davis alum ... and just want to say thanks to her for giving me a video I can share with others that says more about me and my lab than almost anything I have written.
Friday, August 19, 2011
I'd like some open science please
Check out some of the videos from last year's Open Science Summit. I like the first one below, especially since I am wearing my SciFoo shirt. I wore it because many of the people had to leave the Open Science meeting early to go to SciFoo and I was not invited ... do you hear that SciFoo organizers?
Hat tip to Khadijah M. Britton
Hat tip to Khadijah M. Britton
Friday, August 12, 2011
Open Parkinson's - need for more progress on Open Access for papers on Parkinson's
I have a new aspect of open science I am concerned about. It is research on Parkinson's. This comes from having a family member with Parkinson's who is not doing so well. And I am getting frustrated with trying to read up on some Parkinson's related research but hitting a paywall for much of it. Uggh. This is so incredibly annoying and, in many ways, immoral. If we want to speed up the work towards cures, treatments and prevention strategies, we need to share more broadly everything we know about this disease. Yes, I know - publishing costs money. And yes, I know, open access and open science does not in any way solve all the problems associated with scientific and medical discovery and research. But I do know that openness can accelerate the pace of progress.
If you look at "openness" in relation to Parkinson's studies it is pretty disappointing. For example, of 60,400 publications that come up in Pubmed with a simple search for "Parkinson's" 9053 have free full text available. That's is just not enough. One thing that is interesting (and disconcerting) is what it looks like over time:
Now mind you, I have only focused here on availability of free full text. There is another major issue in "open access" which is, well, openness. Openness of papers involves the rules on use, distribution and reuse. Many of the "free full text" papers in Pubmed Central have restrictions on how those papers can be used. The ideal to me is for the papers to be fully open such that anyone can redistribute them, repost them, take material from them, and use it in any way (with attribution). Why is this important? Well, it would allow anyone to create a Parkinson's papers database, for example, with interesting search and discovery features. Or anyone could use a paper as part of a course, without having to worry about permissions. Or anyone could use material from a paper as part of a news story or blog post without getting into fights about fair use.
Now - there are some Parkinson's related papers available out there outside of Pubmed, in institutional repositories and such. But these are not easy to find at this time. So in a way they don't count for much.
I will be writing more about Open Science issues and Parkinson's in the future. But for now I have a request. Anyone out there working on Parkinson's please find a way to make your discoveries more freely and openly and rapidly available. I realize many/most of you are very passionate about this disease. I realize many of you work your tails off. And openness may be the last thing on your mind when working, struggling to get grants and make discoveries and deal with all your other job requirements and life issues. But even if you for some reason choose not to publish in an open access manner, at least post your papers on a website. Deposit them into repositories. Create a Mendeley account and post your papers there. Try and find a way to get them into Pubmed Central. Try to find ways to get your papers picked up by Google Scholar. Do something to increase the probability that your work will have an impact.
I will be writing more about open science and Parkinson's in the future. Stay tuned.
Monday, June 13, 2011
Yes, I am a #RedSox & #PLoS fan; & this video sort of is proof #BenFranklinAward #OpenScience
Just saw this posted on Youtube. Did not know it was coming ... but am happy they recorded it
And here are the slides I used. Will try to synch
For more on this award see
And here are the slides I used. Will try to synch
For more on this award see
Tuesday, April 26, 2011
Lawrence Lessig on Copyright "The architecture of access to scientific knowledge" #CERN
Not much to say other than that Lawrence Lessig is worth listening to.
The Architecture of Access to Scientific Knowledge from lessig on Vimeo.
Lecture at CERN, Geneva, Switzerland, 18 April 2011: A new talk about open access to academic or scientific information, with a bit of commentary about YouTube Copyright School.
Hat tip to Russell Neches for pointing this video out to me ...
The Architecture of Access to Scientific Knowledge from lessig on Vimeo.
Lecture at CERN, Geneva, Switzerland, 18 April 2011: A new talk about open access to academic or scientific information, with a bit of commentary about YouTube Copyright School.
Hat tip to Russell Neches for pointing this video out to me ...
Sunday, April 17, 2011
Boston, Bioinformatics & Ben Franklin Award wrap up from #BioIT11
 |
| Photo by Mark Gabrenya |
Flew to Boston early Tuesday AM. Only thing of note - during Layover in Chicago I saw a bookstore selling autographed versions of "The Immortal Life of Henrietta Lacks" by Rebecca Skloot.
Dropped off my stuff at the Seaport Hotel - had a nice view from my room.
Called up my friend Ashlee Earl who currently works at the Broad and arranged to meet her at Kenmore Square in an hour. I collaborated with Ashlee many years ago on analyzing her expression studies of the Deinococcus radiodurans genome and have been friends ever since. Took the T to Kenmore Square and met Ashlee and then went into this "Fenway Park" place to see a baseball game. (I was born in Boston and am a Redsox fan ...) Had the best baseball seats ever - front row Green Monster Seats - which I had bought from Stubhub.com. Watched the Sox lose while Ashlee and I discussed Genome Centers. I note - to those in the Broad Public Affairs office, Ashlee makes the Broad sound like a great place to work. I tried to get some dirt out of her but she did not provide much.
Took the T back to the hotel after the game. And went to sleep. Got up very early to think about my "acceptance speech" for when I was to pick up my Ben Franklin Award. I made some quick slides on my Ipad (this was the first time I have gone to a meeting w/o my laptop) and during the talk before the award ceremony I emailed them to one of the organizers and we got things set up.
 |
| Photo by Ashlee Earl |
 |
| Photo by Ashlee Earl |
Then I was Introduced and Jeff Bizzaro read a mini statement about why I won the award. Something like what they put on the Bioinformatics.org web site:
Jonathan uses his high visibility in social media to advocate for open access by sharing links to discussions, mentioning open access articles and initiatives, and pushing for the opening up of popular closed access articles. This culture is shared with his students, who advocate for "open access" peer reviewing and created a peer-to-peer service for sharing bioinformatics material (articles, software and datasets). He is the academic editor in chief of PLoS Biology [1] and voices his opinions and support for open access publication and open data sharing on his "Tree of Life" blog [2]. In addition to just voicing his opinion, he also practices what he preaches, by refusing to publish in non-open access journals. With respect to bioinformatics, he has been involved with many software packages that are freely available, such as the recent AMPHORA [3] and PhyloOTU [4]. Lastly, Jonathan helped release a new open data sharing tool for scientists called BioTorrents [5]. This is just another step in encouraging all scientists to share their data and results more openly.
References:
1. http://www.plosbiology.org/
2. http://phylogenomics.blogspot.com/
3. http://genomebiology.com/2008/9/10/R151
4. http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1001061
5. http://www.plosone.org/article/info:doi/10.1371/journal.pone.0010071
Note - am proud to get this award. It is given for contributions to Open Science and previous winners are an esteemed crew: Michael Eisen (my brother), Alex Bateman, Michael Ashburner, Jim Kent, Robert Gentleman, Phil Bourne, Lincoln Stein, Ewan Birney, and Sean Eddy (see full details here).
Then I gave my mini talk focusing on a brief history of how I got into Open Science. Here are my slides
Note the awkward typo where I introduced the "Public Library of Science". Oops. Anyway - talked for a few minutes. While wearing my RedSox PLoS 1 shirt I note.
 |
| Photo by Jeff Bizarro |
 |
| Photo by Mark Gabrenya |
|
| Photo by Mark Gabrenya |
 |
| Photo by Mark Gabrenya |
 |
| Photo by Mark Gabrenya |
 |
| Photo by Mark Gabrenya |
And then back to the meeting where I wandered around again for a while. Saw an old friend from TIGR Xiaoying Lin who now works at Life Technologies and discussed the Ion Torrent with him.
Was pleased to see a booth giving away free RedSox tickets as a prize.
Then I headed out to Brookline for dinner with my Aunt and Uncle and cousin and eventually made my way back to the hotel where I had a few drinks.
The next day I got up a bit late, and eventually made my way to Logan Airport where the trip home was a disaster. My outbound flight was late. Missed my connection. Then the flight I was on was held up for others to make their connection. Though I did get a few hours in Denver Airport to wander around. Got home after 1 AM ... And finally made it home.
Tuesday, March 15, 2011
I know - Ego Blogging is so 2010 - But I won. I won. I won. (The Ben Franklin Award ...)
OK so the title is a bit much. But I am really happy that I won this years Benjamin Franklin Award, given out by the Bioinformatics Organization. For more on this see ...
- UC Davis’ Jonathan Eisen Wins 2011 Benjamin Franklin Award - Bio-IT World
- BioMed Central Blog : Jonathan Eisen: 2011 Benjamin Franklin Laureate
- Jonathan Eisen named 2011 Benjamin Franklin Laureate - DOE Joint ...
I found out a few days ago and am rearranging some things to go to Boston April 13 for the award ceremony at the Bio-IT World Conference and Expo.
From the Bioinformatics Organization web site:
Benjamin Franklin (1706-1790) was one of the most remarkable men of his time. Scientist, inventor, statesman, he freely and openly shared his ideas and refused to patent his inventions. It is the opinion of the founders of the Bioinformatics Organization, Inc. that he embodied the best traits of a scientist, and we seek to honor those who share these virtues
The Benjamin Franklin Award for Open Access in the Life Sciences is a humanitarian/bioethics award presented annually by this organization to an individual who has, in his or her practice, promoted free and open access to the materials and methods used in the life sciences.
I like the general sentiment very much. And perhaps more important - the list of prior winners is an impressive crew. Again, from the Bioinformatics Organization web site:
Note - my brother won the first one.
- 2010 - Alex Bateman
- 2009 - Philip E. Bourne
- 2008 - Robert Gentleman
- 2007 - Sean Eddy
- 2006 - Michael Ashburner
- 2005 - Ewan Birney
- 2004 - Lincoln D. Stein
- 2003 - James Kent
- 2002 - Michael B. Eisen
Anyway - am thinking about what to say in the awards ceremony. Probably going to say something about how openness is more than about being at no charge. Also I might discuss how it would be good to have a female winner one of these days. Speaking of which - maybe people can give suggestions for women to nominate for next year ...
Subscribe to:
Posts (Atom)
