Difference between revisions of "Transition probabilities from selected texts"
(Added hmmer score) |
(Explanation of HMMER score) |
||
| Line 2: | Line 2: | ||
For transitions that have p=0, corrections to p=0.0001 have been performed to attain a non-zero Markov probability. | For transitions that have p=0, corrections to p=0.0001 have been performed to attain a non-zero Markov probability. | ||
| + | |||
| + | HMMER score is the log (base 2) of Markov probability / null probability (1/26^44) | ||
==First order== | ==First order== | ||
| Line 8: | Line 10: | ||
<br/>Markov Probability: 1.4641414719132942E-71 | <br/>Markov Probability: 1.4641414719132942E-71 | ||
<br/>Corrected Zeroes: 1 | <br/>Corrected Zeroes: 1 | ||
| − | <br/> | + | <br/>HMMER Score: -28.487492178774602 |
<br/> | <br/> | ||
<br/>(Les Orientales - Victor Hugo.txt) All Letters | <br/>(Les Orientales - Victor Hugo.txt) All Letters | ||
<br/>Markov Probability: 1.1571661202766167E-78 | <br/>Markov Probability: 1.1571661202766167E-78 | ||
<br/>Corrected Zeroes: 2 | <br/>Corrected Zeroes: 2 | ||
| − | <br/> | + | <br/>HMMER Score: -52.08044781349958 |
<br/> | <br/> | ||
<br/>(Traumdeutung - Sigmund Freud.txt) All Letters | <br/>(Traumdeutung - Sigmund Freud.txt) All Letters | ||
<br/>Markov Probability: 3.866259362091221E-77 | <br/>Markov Probability: 3.866259362091221E-77 | ||
<br/>Corrected Zeroes: 1 | <br/>Corrected Zeroes: 1 | ||
| − | <br/> | + | <br/>HMMER Score: -47.018177286334875 |
<br/> | <br/> | ||
<br/>(Vigenere - 1984.txt) All Letters | <br/>(Vigenere - 1984.txt) All Letters | ||
<br/>Markov Probability: 1.646391769425068E-70 | <br/>Markov Probability: 1.646391769425068E-70 | ||
<br/>Corrected Zeroes: 0 | <br/>Corrected Zeroes: 0 | ||
| − | <br/> | + | <br/>HMMER Score: -24.99631136880728 |
<br/> | <br/> | ||
===Initial letters=== | ===Initial letters=== | ||
| Line 29: | Line 31: | ||
<br/>Markov Probability: 1.9187432339606176E-56 | <br/>Markov Probability: 1.9187432339606176E-56 | ||
<br/>Corrected Zeroes: 0 | <br/>Corrected Zeroes: 0 | ||
| − | <br/> | + | <br/>HMMER Score: 21.731535947650737 |
<br/> | <br/> | ||
<br/>(Les Orientales - Victor Hugo.txt) Initial Letters | <br/>(Les Orientales - Victor Hugo.txt) Initial Letters | ||
<br/>Markov Probability: 7.809561685705767E-61 | <br/>Markov Probability: 7.809561685705767E-61 | ||
<br/>Corrected Zeroes: 0 | <br/>Corrected Zeroes: 0 | ||
| − | <br/> | + | <br/>HMMER Score: 7.14697538897068 |
<br/> | <br/> | ||
<br/>(Traumdeutung - Sigmund Freud.txt) Initial Letters | <br/>(Traumdeutung - Sigmund Freud.txt) Initial Letters | ||
<br/>Markov Probability: 4.553994899282612E-68 | <br/>Markov Probability: 4.553994899282612E-68 | ||
<br/>Corrected Zeroes: 1 | <br/>Corrected Zeroes: 1 | ||
| − | <br/> | + | <br/>HMMER Score: -16.88463017855261 |
<br/> | <br/> | ||
==Second order== | ==Second order== | ||
| Line 46: | Line 48: | ||
<br/>Markov Probability: 2.115089006082555E-99 | <br/>Markov Probability: 2.115089006082555E-99 | ||
<br/>Corrected Zeroes: 14 | <br/>Corrected Zeroes: 14 | ||
| − | <br/> | + | <br/>HMMER Score: -120.970815420271 |
<br/> | <br/> | ||
<br/>(Les Orientales - Victor Hugo.txt) All Letters | <br/>(Les Orientales - Victor Hugo.txt) All Letters | ||
<br/>Markov Probability: 4.429249667205306E-106 | <br/>Markov Probability: 4.429249667205306E-106 | ||
<br/>Corrected Zeroes: 18 | <br/>Corrected Zeroes: 18 | ||
| − | <br/> | + | <br/>HMMER Score: -143.15796813874405 |
<br/> | <br/> | ||
<br/>(Traumdeutung - Sigmund Freud.txt) All Letters | <br/>(Traumdeutung - Sigmund Freud.txt) All Letters | ||
<br/>Markov Probability: 3.796449095384246E-119 | <br/>Markov Probability: 3.796449095384246E-119 | ||
<br/>Corrected Zeroes: 21 | <br/>Corrected Zeroes: 21 | ||
| − | <br/> | + | <br/>HMMER Score: -186.56544502943777 |
<br/> | <br/> | ||
<br/>(Vigenere - 1984.txt) All Letters | <br/>(Vigenere - 1984.txt) All Letters | ||
<br/>Markov Probability: 1.669944098510842E-92 | <br/>Markov Probability: 1.669944098510842E-92 | ||
<br/>Corrected Zeroes: 8 | <br/>Corrected Zeroes: 8 | ||
| − | <br/> | + | <br/>HMMER Score: -98.05823732223358 |
<br/> | <br/> | ||
===Initial letters=== | ===Initial letters=== | ||
| Line 67: | Line 69: | ||
<br/>Markov Probability: 7.009981410871375E-61 | <br/>Markov Probability: 7.009981410871375E-61 | ||
<br/>Corrected Zeroes: 2 | <br/>Corrected Zeroes: 2 | ||
| − | <br/> | + | <br/>HMMER Score: 6.991144428568879 |
<br/> | <br/> | ||
<br/>(Les Orientales - Victor Hugo.txt) Initial Letters | <br/>(Les Orientales - Victor Hugo.txt) Initial Letters | ||
<br/>Markov Probability: 2.9707877167599384E-89 | <br/>Markov Probability: 2.9707877167599384E-89 | ||
<br/>Corrected Zeroes: 12 | <br/>Corrected Zeroes: 12 | ||
| − | <br/> | + | <br/>HMMER Score: -87.26140732840628 |
<br/> | <br/> | ||
<br/>(Traumdeutung - Sigmund Freud.txt) Initial Letters | <br/>(Traumdeutung - Sigmund Freud.txt) Initial Letters | ||
<br/>Markov Probability: 2.9078792518323414E-100 | <br/>Markov Probability: 2.9078792518323414E-100 | ||
<br/>Corrected Zeroes: 17 | <br/>Corrected Zeroes: 17 | ||
| − | <br/> | + | <br/>HMMER Score: -123.83349452718522 |
<br/> | <br/> | ||
| − | |||
==See also== | ==See also== | ||
Revision as of 18:11, 15 June 2009
The Somerton Man's code (without the extra line) is 44 characters long. So, if the text is purely random (1/26 chance of each letter appearing) then the probability of attaining this particular string of 44 is (1/26)^44 = 5.51027E-63. This is a good initial comparison.
For transitions that have p=0, corrections to p=0.0001 have been performed to attain a non-zero Markov probability.
HMMER score is the log (base 2) of Markov probability / null probability (1/26^44)
Contents
First order
All letters
(1984 - George Orwell.txt) All Letters
Markov Probability: 1.4641414719132942E-71
Corrected Zeroes: 1
HMMER Score: -28.487492178774602
(Les Orientales - Victor Hugo.txt) All Letters
Markov Probability: 1.1571661202766167E-78
Corrected Zeroes: 2
HMMER Score: -52.08044781349958
(Traumdeutung - Sigmund Freud.txt) All Letters
Markov Probability: 3.866259362091221E-77
Corrected Zeroes: 1
HMMER Score: -47.018177286334875
(Vigenere - 1984.txt) All Letters
Markov Probability: 1.646391769425068E-70
Corrected Zeroes: 0
HMMER Score: -24.99631136880728
Initial letters
(1984 - George Orwell.txt) Initial Letters
Markov Probability: 1.9187432339606176E-56
Corrected Zeroes: 0
HMMER Score: 21.731535947650737
(Les Orientales - Victor Hugo.txt) Initial Letters
Markov Probability: 7.809561685705767E-61
Corrected Zeroes: 0
HMMER Score: 7.14697538897068
(Traumdeutung - Sigmund Freud.txt) Initial Letters
Markov Probability: 4.553994899282612E-68
Corrected Zeroes: 1
HMMER Score: -16.88463017855261
Second order
All letters
(1984 - George Orwell.txt) All Letters
Markov Probability: 2.115089006082555E-99
Corrected Zeroes: 14
HMMER Score: -120.970815420271
(Les Orientales - Victor Hugo.txt) All Letters
Markov Probability: 4.429249667205306E-106
Corrected Zeroes: 18
HMMER Score: -143.15796813874405
(Traumdeutung - Sigmund Freud.txt) All Letters
Markov Probability: 3.796449095384246E-119
Corrected Zeroes: 21
HMMER Score: -186.56544502943777
(Vigenere - 1984.txt) All Letters
Markov Probability: 1.669944098510842E-92
Corrected Zeroes: 8
HMMER Score: -98.05823732223358
Initial letters
(1984 - George Orwell.txt) Initial Letters
Markov Probability: 7.009981410871375E-61
Corrected Zeroes: 2
HMMER Score: 6.991144428568879
(Les Orientales - Victor Hugo.txt) Initial Letters
Markov Probability: 2.9707877167599384E-89
Corrected Zeroes: 12
HMMER Score: -87.26140732840628
(Traumdeutung - Sigmund Freud.txt) Initial Letters
Markov Probability: 2.9078792518323414E-100
Corrected Zeroes: 17
HMMER Score: -123.83349452718522
