ITIL 3.7.19 Password Storage Password Storage: 1. A simple rule is that a password must never be stored in any database.   However a hash of the password must be stored so it can be verified when a person signs in. 2. Unfortunately NSA have been part of the International Standards for the way that languages are permitted to hash a password.   It is suggested that many such hash methods are reversible for modest length passwords.   Hash reversal is done using a Rainbow table - a list of possible passwords that will generate the hashed value. Caesar Method: 1. The frequency of the letters "ETAOI" (both cases) in the English language are unreasonably high and can be used to guess a persons password.   A simple encryption method is to drop an instance of these letters so they are never stored and cannot be used to decrypt a password.   The frequency of the first letter in an English word means that password reversal has real merits - "T" is the first letter in 17% of English words - "A" is 12%.   The elimination of "TA" in a password will dramatically reduce the power of quantum computers to intelligently guess a password from a hash. 2. The reverse is also true, the least used letters in English words are easy to identify as "ZQXJ" - when these exist in a hash table, then decryption becomes easier.   High usage valid letters include "NSHRDL" and low usage valid letters include "CUMWFGYPBVK". 3. Observation: Each language written using Latin character codes has its own unique distribution of letters that enable the language to be identified.   For example: the Germans will use four-times more "G" letters than the French, but the letter "P" is used four-times more by the French than the Germans. Decryption: 1. Millions of learned people in the world have studied for 2000 years on how the decrypt encrypted information.   It is likely that today, more than one million very intelligent people are studying the RSA encryption method to see how it can be broken using very fast bespoke quantum computers.   The same is true of any published encryption method - if a programming language supports a method, people can gain great benefits from learning how to crack the encryption.   It may take one million people five or more years to crack some of the encryption methods in use today, but it can be guaranteed that one day they will be broken, but such news may never be made public. 2. Frequency distribution is a key decryption method - the likelihood of one letter following another letter is somewhat predictable.   The first rule of encryption is to break any frequency distribution analysis - for example: reduce the distribution of 26 letters down to the distribution of 10 numbers.   Frequency distribution of numbers begins with very few numbers beginning with a zero, unless they are a telephone number- easy decryption methods. 3. Letter substitution tables do not work because the frequency distribution remains, no matter what code replaces the letter.   But where multiple parts of a letter and multiple letters are substituted, then thousands of years of frequency distribution can be foiled.   Where letters are transformed into numbers and then algorithms transform those numbers, then the resulting numbers will not be reversed by any powerful substitution computer. Letter Substitution: 1. It is not practical to hide "ETAOI" using any substitution method, however multiple layers substitution can destroy frequency distribution.   All letters can be shifted up one letter with "Z" wrapped to become "A" - the result is then encrypted using a standard method that will find the "F" is the highest used letter. 2. Letter-pair substitution is more extreme - the top 99 letter-pairs are substituted using one method and remaining letters are substituted using a different method.   Word and space substitution is even better - a complete common word with its preceding space is substituted for a code - " the" is 21 Solution: 1. Information and intelligence on the limitations of password hashing methods that are built into common programming languages is the start.   As every encryption method ever invented has eventually been broken, a critical rule is to use more than one encryption method.   When layer upon layer of encryption method is used, any attempts to decrypt will not know when they have finished different layers.   The solution is NOT to rely on any one encryption method that is so well known that it is built into the programming language. Reverse and Insert: 1. When a password is entered, it is purposefully changed before it is stored - what is stored is not decryptable to the password that a person enters. Example: as user password is "password123456" - this is as weak as they can get, but it is 14 bytes. Fragment: the password into 3 byte groups such as "pas", "swo", "rd1", "234", "56" - five parts. Reverse: each 3 byte group. Append: a symbol (or letter) at the end of each group to make it 4 bytes. Assemble: the result as "sap+ows-1dr)432(65." as 19 bytes - the order of the groups may be scrambled. Hash: the result and store it in the database as normal. Benefit: 1. When an agency takes a copy of the database, they can use Rainbow tables in powerful quantum computers to attempt to decrypt the password into a likely set of passwords.   Whatever results are decrypted will be wrong because the agency is unable to know what extra encryption methods have been used.   It is not the hashing that keeps the data safe and secure, its the extra layers of encryption. Morse Code: 1. An encryption method is to use Morse code with 3 symbols as 1 as dot, 2 as dash and 0 as space.   The resulting string of base 3 digits has a frequency distribution based on common letters having less parts than less common letters.   Using the Philips Abbreviation code to replace common words and phrases, Morse code can deliver a two way encryption method that a modern generation may not comprehend.   User Name "Tom Jones" may be stored as "2022202200122202220210101110" - in reality bytes would be reversed and scrambled.   Frequency distribution only applies after the agency has figured out that variable length letter substitution is employed.   Morse code encryption is suitable for an address where upper and lower case can be deduced by capitalization.   "Watford" may be reversed and stored as "2191081271062252041132190280172061252" - in practice, triple-interleaved scrambling would be used.   Any other digits in the range 3 to 9 may be randomly generated and inserted to obfuscate - decrypt will simply ignore invalid digits. Pass Phrase is Not Stored: 1. The business has a first class solution where the pass phrase is not stored, but calculated from a lot of other data when needed.   The business has chosen to assign a pass phrase to each person because the pass phrase is far to important to be left to the whim of a person where evidence is that they are likely to choose 123456.   Because the business has to calculate the pass phrase, the business is able to do that calculation each time the pass phrase is needed. 2. The benefit of not having to store the pass phrase has considerable benefits, none less than the pass phrases cannot be stolen.   The complexity of the algorithm used to calculate the pass phrase is legendary and continues to evolve as improved methods are identified.   As the algorithm gets more sophisticated, existing pass phrases continue to be used, but the later pass phrases are designed to a higher level of obfuscation. 3. The length and format of the pass phrase defines the edition of the algorithm used.   The only check to be done on an entered pass phrase is to question if it is correct for the approved person - all other results mean the pass phrase is rejected. 4. Criminals and agencies go to a lot of trouble to root out passwords, so a pseudo password is stored, but it is invalid and not used.   The concept of a criminals stealing made up passwords that do not work has an appeal and can be easily be identified if one of these pseudo passwords is entered.   The pseudo password is like a calling card to say "I am a criminal please put a block on this computer". Deniable Data Design: 1. Encrypting a password is not good enough - all associated data must also be encrypted.   A criminal able to crack a password algorithm is worthless if all associated data is also encrypted using different encryption methods.   When every stored field is just a number, then it is hard to say what is what. 2. Entire tables that actually store sign-in data can be made to look like worthless reference data such as a list of three million company names or popular baby names by year and country or pictures of the countryside.   The concept of deniable data is based on laws that can demand that a system administrators hand over their password and encryption key.   The first solution is to eliminate the role of system administrator and the second is to replace the encryption key with many millions of encryption keys that are hidden in a forest of encryption keys. 3. A standard encryption key is 2048 bits as 256 bytes or 350 characters.   A standard image is 10,000 characters and a standard picture is 1,000,000 characters.   It can be very easy to assign an encryption key as 350 characters from some offset within a picture in a folder holding millions of pictures.   It is plausible to say that the encryption key does not exist, because the picture existed before the encryption key existed. Deniable Date: 1. Every date must be stored as an integer number less than 11 decimal digits that is sortable and filterable.   The algorithm to encrypt a date is to derive the unit number of seconds from a prior known event.   Each encrypted date may have its own unique prior known event that is known to the business and stored in a public picture. 2. An objective is to make sure that each stored date does not look like a date, but to look like a meaningless number that could be a token to any field value.   Decryption of the date is dependent on another prior event date that cannot be known by a criminal. 3. Dates tend to be historical (has happened) or future (is scheduled to happen). A non-linear algorithm is used to deduce a token from a date.   Historical dates may be stored as a unit number of seconds before the record was created.   Future dates may be stored as the a unit number of seconds after the record was created.   Creation dates may be stored as a unit number of seconds after the business was created where the units may be 1234 seconds. Deniable Email: 1. Every email must be stored as a set of integer numbers.   Because the domain or suffix is common, it is tokenised as a number from an option list.   Each encrypted date may have its own unique prior known event that is known to the business and stored in a public picture. 2. Business email address and private email address may be the same or may be different.   Private email is used to sign-in and by the business to communicate secrets to the person - the email may be a fake.   A private fake email address may be forwarded by the business to a real email address.   Business email is used in standard documents to communicate with customers and suppliers using encrypted communications. HRM WOP: UAD exists as an index to all locations that are relevant to the business. UAD38. address country UAD50. company registration country UAD48. time zone UAD54. base currency UAP is a person and all the things that represent that person, regardless of UAD. UAP35. contact by UAP36. language UAP38. address country UAP48. time zone UAP50. job function UAP58. department UAS is a deniable security concept with one or many for each UAP. UAS27. hours of day UAS28. days of week UAS29. check UAS30. expiry days UAS57. welcome page UAS59. role UAS61. geo-location UAS63. ISP UAS33. screen width UAS34. colour resolution UAS52. computer capacity UAS53. network capacity UAS54. processor UAS55. operating system UAS56. browser HRM Date: UAP54. last sin date + implied expiry date UAP57. last sout date UAP52. last add new date UAP53. last change date UAP55. earliest sin time UAP53. latest sin time UAP27. last weekend sin date UAP28. left date Document Control: 2016 Sep 16 : Latest edition as (public) page 163719     Part of common ITIL application service. SQL and Tokens: 1. U33 stores person records.   c01 is pkey. c23 is first name token. c24 is family name token. c25 is phone number. c26 is branch. c27 is job title. 2. U34 stores tokens and text for any record-field.   c01 is pkey. c02 is record. c03 is field number. c04 is list sequence number. c05 is token. c06 is text value of up to 32 characters.   POLICY: token c05 is not unique like pkey c01, it is purposefully obfuscated like a date 20160303 with the same token being reused for different fields. 3. Default: SELECT * FROM u33 WHERE 1=1 ORDER BY u33.c01 ASC LIMIT 0,16. 4. First "T" Filter: SELECT * FROM u33 JOIN u34 ON u33.c23=u34.c05 WHERE 1=1 AND u34.c02="u33" AND u34.c03="23" AND u34.c06 LIKE "T%" ORDER BY u34.c05 ASC LIMIT 0,16. 5. As "T" Filter: SELECT * FROM u33 JOIN u34 AS j23 ON u33.c23=j23.c05 WHERE 1=1 AND j23.c02="u33" AND j23.c03="23" AND j23.c06 LIKE "T%" ORDER BY j23.c05 ASC LIMIT 0,16. 6. Family "W" Filter: SELECT * FROM u33 JOIN u34 AS j23 ON u33.c23=j23.c05 JOIN u34 as j24 ON u33.c24=j24.c05 WHERE 1=1 AND j23.c02="u33" AND j23.c03="23" AND j23.c06 LIKE "T%" AND j24.c02="u33" AND j24.c03="24" AND j24.c05 LIKE "W%" ORDER BY j23.c06 ASC, j24.c06 ASC LIMIT 0,16. 7. STOP: While the use of tokens is a step towards improved security, this SQL JOIN performance cost in a large table may not be an effective solution.   The design option to use tokens to replace all text fields in all records is unlikely to be a viable security policy. Not all SQL left join syntax is identical.   This is SQL evidence of why ordinary people do not use pseudonymised tokens. Pseudonymisation: 1. A more intelligent pre-processed array may be more effective and more scaleable. 2. Default: SELECT * FROM u33 WHERE 1=1 ORDER BY u33.c01 ASC LIMIT 0,16. 3. First "T" Filter: SELECT c05,c06 FROM u34 WHERE 1=1 AND c02="u33" AND c03="23" AND c06 LIKE "T%" ORDER BY c06 ASC LIMIT 0,16 = result into array a23. 4. Family "W" Filter: SELECT c05,c06 FROM u34 WHERE 1=1 AND c02="u33" AND c03="24" AND c06 LIKE "W%" ORDER BY c06 ASC LIMIT 0,16 = result into array a24. 6. Spreadsheet: SELECT * FROM u33 WHERE 1=1 AND (c23="a23.0" OR c23="a23.1" OR c23="a23.2" OR ...) AND (c24="a24.0" OR c24="a24.1" OR c24="a24.2" OR ...) ORDER BY c23 ASC, c24 ASC LIMIT 0,16.   Where the SQL where clause is created to identify up to 16 potential token values for each filtered field. 7. The token and its text value are stored in an array so the text value can be shown in the spreadsheet display.   This minimises the SQL lookup requirements and enables a larger number of fields to be filtered.   STEGANOGRAPHY: Tokens and text values may be encrypted as a number string and hidden in an image that is unique to each field. 8. Sort and filter are processed in the same way where the text filter value may be ommitted.   Sort needs up to 16 ordered token values to be selected from the main table.   Preprocessing is optional and only needed when a sort order or filter has been requested for a tokenized field value. 9. ENCRYPTOGRAPHY: acts on the text value to store it as a number string.   Record code is encrypted as a number and concatenated with field number (c02+c03).   Sequence is optional and only used when a drop down list must be ordered by use, rather than text value. User Preference: 1. A fully design cycle is planned to return to a database driven user preference for each spreadsheet to replace the current cookie solution.   Cookies have been exceeded on a few occations and they continually increase page download times.   The alternative of "Local Storage" has been rejected because SQL is more effective that downloading stored data from a local computer. 2. z333 is a bespoke user preference table for each spreadsheet using a similar storage string as cookies.   Select by the (session) user-key and the (c2_url) page-number. Also manage version, last-used-date and used-counter.   At any time, a record can be physically deleted and it shall be recreated the next time the user opens that spreadsheet page.   Where the last used date exceeds xxx number of days, then the record may be reset to its default value. 3. c01 auto incremented key. c02 user-key (index). c03 page-number (index). c04 last-used-date. c05 used-counter. c06 mode-number. c07 rows-per-page. c08 page-number. c09 version, c10 dot-separated-field-array (512)   Array as: MAX_ROWS, PAGE_NUMB, MODE, AAA_FIELD. 4. Every spreadsheet has a version number 0-9 stored as a ties value - this will randomly change from time to time.   When the user preference is selected, if the version is not equal, delete/reset user preference.   When the user preference is selected, if the last-used-date is less than today minus expired days, delete/reset user preference. 5. Steps: (1) Get z333. (2) Sort. (3) Hide-show. (4) Left-right. (5) Filter. (6) Where. (7) Order-by. (8) Paging-next-back. (9) Paging-count-rows. (10) Put z333.   A minor continual improvement that will not be noticed by users, but will give a decade of performance and quality benefits.