Recommandé
Recommandé
Contenu connexe
En vedette
En vedette (19)
Similaire à Bottoms bosc2010 bio_snp_inherit
Similaire à Bottoms bosc2010 bio_snp_inherit (13)
Plus de BOSC 2010
Plus de BOSC 2010 (20)
Dernier
Dernier (20)
Bottoms bosc2010 bio_snp_inherit
- 1. SNP Allele Designations (Bio::SNP::Inherit) Christopher Bottoms BOSC 2010
- 2. 5 million data “items” one CPU: 2+ days eight CPUs: 1-2 days
- 3. SNP ID Sample ID Base1 Base2 1 1 A A 1 2 A A 1 3 A G … … … … 1 5000 A A 2 1 C C … … … … … … … … 1106 5000 GG GG
- 4. SNP ID Sample ID Base1 Base2 1 1 A A 1 2 A A 1 3 A G … … … … 1 5000 A A 2 1 C C … … … … … … … … 1106 5000 GG GG
- 5. “ Matrix” data file format SNP ID 1 2 3 … 5000 SNP1 AA AA AG … AA SNP2 CC GG GG … CG
- 6. “ Matrix” data file format SNP ID 1 2 3 … 5000 SNP1 AA AA AG … AA SNP2 CC GG GG … CG
- 8. ID’s file ID Name Group 1 B73 B73 2 B73xZ1 NAMF1 3 Mo17 Control 4 M100 IBM 5 Bob B73xZ1
- 9. ID’s file ID Name Group 1 B73 B73 2 B73xZ1 NAMF1 3 Mo17 Control 4 M100 IBM 5 Bob B73xZ1
- 10. “ Human Parsed” ID’s file ID Name Group A (ID) B (ID) AxB (ID) 1 B73 B73 2 B73xZ1 NAMF1 3 Mo17 Control 4 M100 IBM 1 3 5 Bob B73xZ1 1 2
- 14. End
Notes de l'éditeur
- The data file had to read into the database and then the information from the database was used to determine inheritance codes.
- We had 5000 samples of data associated with one “SNP ID” and we had over 1000 SNP ID’s, making our data file over 5 million lines long. It was actually much messier looking than this and I ended up processing each line and storing the results in a database. After talking with my boss about this, he provided me the same data in a different format.
- We had 5000 samples of data associated with one “SNP ID” and we had over 1000 SNP ID’s, making our data file over 5 million lines long. It was actually much messier looking than this and I ended up processing each line and storing the results in a database. After talking with my boss about this, he provided me the same data in a different format.
- This format really condensed the data file. From 800MB to less than 15MB, in fact. However, now each “data point” isn’t “tagged”, so some additional preprocessing needed to be done.
- This format really condensed the data file. From 800MB to less than 15MB, in fact. However, now each “data point” isn’t “tagged”, so some additional preprocessing needed to be done.
- The sample ID’s I showed you earlier, each represented a different individual corn plant. Knowing the relationships among the different plants was required for processing the data. Here, since I’m a human familiar the genetic system, I know that IBM stands for an Intermated B73 x Mo17 population. This is a simplified example of a manifest file. Z1, M100, and “Bob” are just made up names and any similarity to known names is purely coincidental. When you start looking at these, you see that the way the Relationships were defined in multiple ways. There isn’t anything here that directly tells that IBM and Mo17 and B73 are related. To take advantage of this information I wrote a long series of rules. Well, the break through came with the realization that I couldn’t keep this up forever. Instead of telling the computer how to understand these relationships, I decided to just tell the computer what the relationships are (next slide).
- The sample ID’s I showed you earlier, each represented a different individual corn plant. Knowing the relationships among the different plants was required for processing the data. Here, since I’m a human familiar the genetic system, I know that IBM stands for an Intermated B73 x Mo17 population. This is a simplified example of a manifest file. Z1, M100, and “Bob” are just made up names and any similarity to known names is purely coincidental. When you start looking at these, you see that the way the Relationships were defined in multiple ways. There isn’t anything here that directly tells that IBM and Mo17 and B73 are related. To take advantage of this information I wrote a long series of rules. Well, the break through came with the realization that I couldn’t keep this up forever. Instead of telling the computer how to understand these relationships, I decided to just tell the computer what the relationships are (next slide).
- This is organized in a way that is simple to both humans and computer programs to understand.
- Configuration files are great for some tasks that are easy for humans but more difficult to program. They are also great for things that are variable Setting up the configuration file only takes minutes. If we don’t know what these relationships are to start with, then we’re in trouble anyway. Simple for humans ≠ simple for computers Something else I didn’t put up here is that reducing your dependencies sure makes it easier to install.