Leek: Government Secrets Mac OS

Posted on  by

About the game Secret Government is a grand strategy game about a secret society. Remaining unseen, they have been ruling mankind for hundreds of years. Your role will not be just to lead a particular country or region, but the Brotherhood — a centuries-old organization. The Indian government then 'utilized backdoors provided by RINOA' to intercept internal emails of the U.S.-China Economic and Security Review Commission, a U.S. Government body with a mandate to monitor, investigate and report to Congress on 'the national security implications of the bilateral trade and economic relationship' between the U.S.

PEP:506
Title:Adding A Secrets Module To The Standard Library
Author:Steven D'Aprano <steve at pearwood.info>
Status:Final
Type:Standards Track
Created:19-Sep-2015
Python-Version:3.6
Post-History:

Contents

Install the SEM Agent on macOS X. See SEM Agent pre-installation checklist: Prepare to deploy SEM Agents for Agent download information and a pre-install checklist. SEM does not currently support USB defender on macOS X. Installation notes Installing the SEM Agent on macOS requires enabling the 'root' user account and disabling System Integrity Protection (SIP). Secret Government is a grand strategy game about a secret society. Remaining unseen, they have been ruling mankind for hundreds of years. Your role will be not to just lead any particular country or region, but the Brotherhood: a centuries-old organization that works in the shadows and manipulates the entire world.

  • API and Implementation

This PEP proposes the addition of a module for common security-relatedfunctions such as generating tokens to the Python standard library.

Some common abbreviations used in this proposal:

  • PRNG:

    Pseudo Random Number Generator. A deterministic algorithm usedto produce random-looking numbers with certain desirablestatistical properties.

  • CSPRNG:

    Cryptographically Strong Pseudo Random Number Generator. Analgorithm used to produce random-looking numbers which areresistant to prediction.

  • MT:

    Mersenne Twister. An extensively studied PRNG which is currentlyused by the random module as the default.

This proposal is motivated by concerns that Python's standard librarymakes it too easy for developers to inadvertently make serious securityerrors. Theo de Raadt, the founder of OpenBSD, contacted Guido van Rossumand expressed some concern [1] about the use of MT for generating sensitiveinformation such as passwords, secure tokens, session keys and similar.

Although the documentation for the random module explicitly states thatthe default is not suitable for security purposes [2], it is stronglybelieved that this warning may be missed, ignored or misunderstood bymany Python developers. In particular:

  • developers may not have read the documentation and consequentlynot seen the warning;
  • they may not realise that their specific use of the module has securityimplications; or
  • not realising that there could be a problem, they have copied code(or learned techniques) from websites which don't offer bestpractises.

The first [3] hit when searching for 'python how to generate passwords' onGoogle is a tutorial that uses the default functions from the randommodule [4]. Although it is not intended for use in web applications, it islikely that similar techniques find themselves used in that situation.The second hit is to a StackOverflow question about generatingpasswords [5]. Most of the answers given, including the accepted one, usethe default functions. When one user warned that the default could beeasily compromised, they were told 'I think you worry too much.' [6]

This strongly suggests that the existing random module is an attractivenuisance when it comes to generating (for example) passwords or securetokens.

Additional motivation (of a more philosophical bent) can be found in thepost which first proposed this idea [7].

Alternative proposals have focused on the default PRNG in the randommodule, with the aim of providing 'secure by default' cryptographicallystrong primitives that developers can build upon without thinking aboutsecurity. (See Alternatives below.) This proposes a different approach:

  • The standard library already provides cryptographically strongprimitives, but many users don't know they exist or when to use them.
  • Instead of requiring crypto-naive users to write secure code, thestandard library should include a set of ready-to-use 'batteries' forthe most common needs, such as generating secure tokens. This codewill both directly satisfy a need ('How do I generate a password resettoken?'), and act as an example of acceptable practises whichdevelopers can learn from [8].

To do this, this PEP proposes that we add a new module to the standardlibrary, with the suggested name secrets. This module will contain aset of ready-to-use functions for common activities with securityimplications, together with some lower-level primitives.

The suggestion is that secrets becomes the go-to module for dealingwith anything which should remain secret (passwords, tokens, etc.)while the random module remains backward-compatible.

This PEP proposes the following functions for the secrets module:

  • Functions for generating tokens suitable for use in (e.g.) passwordrecovery, as session keys, etc., in the following formats:
    • as bytes, secrets.token_bytes;
    • as text, using hexadecimal digits, secrets.token_hex;
    • as text, using URL-safe base-64 encoding, secrets.token_urlsafe.
  • A limited interface to the system CSPRNG, using either os.urandomdirectly or random.SystemRandom. Unlike the random module, thisdoes not need to provide methods for seeding, getting or setting thestate, or any non-uniform distributions. It should provide thefollowing:
    • A function for choosing items from a sequence, secrets.choice.
    • A function for generating a given number of random bits and/or bytesas an integer, secrets.randbits.
    • A function for returning a random integer in the half-open range0 to the given upper limit, secrets.randbelow[9].
  • A function for comparing text or bytes digests for equality while beingresistant to timing attacks, secrets.compare_digest.
Leek: government secrets mac os x

The consensus appears to be that there is no need to add a new CSPRNG tothe random module to support these uses, SystemRandom will besufficient.

Some illustrative implementations have been given by Nick Coghlan [10]and a minimalist API by Tim Peters [11]. This idea has also been discussedon the issue tracker for the 'cryptography' module [12]. The followingpseudo-code should be taken as the starting point for the realimplementation:

The secrets module itself will be pure Python, and other Pythonimplementations can easily make use of it unchanged, or adapt it asnecessary. An implementation can be found on BitBucket [13].

Default arguments

One difficult question is 'How many bytes should my token be?'. We canhelp with this question by providing a default amount of entropy for the'token_*' functions. If the nbytes argument is None or not given, thedefault entropy will be used. This default value should be large enoughto be expected to be secure for medium-security uses, but is expected tochange in the future, possibly even in a maintenance release [14].

Naming conventions

One question is the naming conventions used in the module [15], whether touse C-like naming conventions such as 'randrange' or more Pythonic namessuch as 'random_range'.

Functions which are simply bound methods of the private SystemRandominstance (e.g. randrange), or a thin wrapper around such, should keepthe familiar names. Those which are something new (such as the varioustoken_* functions) will use more Pythonic names.

One alternative is to change the default PRNG provided by the randommodule [16]. This received considerable scepticism and outright opposition:

  • There is fear that a CSPRNG may be slower than the current PRNG (whichin the case of MT is already quite slow).
  • Some applications (such as scientific simulations, and replayinggameplay) require the ability to seed the PRNG into a known state,which a CSPRNG lacks by design.
  • Another major use of the random module is for simple 'guess a number'games written by beginners, and many people are loath to make anychange to the random module which may make that harder.
  • Although there is no proposal to remove MT from the random module,there was considerable hostility to the idea of having to opt-in toa non-CSPRNG or any backwards-incompatible changes.
  • Demonstrated attacks against MT are typically against PHP applications.It is believed that PHP's version of MT is a significantly softer targetthan Python's version, due to a poor seeding technique [17]. Consequently,without a proven attack against Python applications, many people objectto a backwards-incompatible change.

Nick Coghlan made an earlier suggestion for a globally configurable PRNGwhich uses the system CSPRNG by default [18], but has since withdrawn itin favour of this proposal.

  • PHP

    PHP includes a function uniqid[19] which by default returns athirteen character string based on the current time in microseconds.Translated into Python syntax, it has the following signature:

    The PHP documentation warns that this function is not suitable forsecurity purposes. Nevertheless, various mature, well-known PHPapplications use it for that purpose (citation needed).

    PHP 5.3 and better also includes a function openssl_random_pseudo_bytes[20]. Translated into Python syntax, it has roughly the followingsignature:

    This function returns a pseudo-random string of bytes of the givenlength, and a boolean flag giving whether the string is consideredcryptographically strong. The PHP manual suggests that returninganything but True should be rare except for old or broken platforms.

  • JavaScript

    Based on a rather cursory search [21], there do not appear to be anywell-known standard functions for producing strong random values inJavaScript. Math.random is often used, despite serious weaknessesmaking it unsuitable for cryptographic purposes [22]. In recent yearsthe majority of browsers have gained support for window.crypto.getRandomValues[23].

    Node.js offers a rich cryptographic module, crypto[24], most ofwhich is beyond the scope of this PEP. It does include a single functionfor generating random bytes, crypto.randomBytes.

  • Ruby

    The Ruby standard library includes a module SecureRandom[25]which includes the following methods:

    • base64 - returns a Base64 encoded random string.
    • hex - returns a random hexadecimal string.
    • random_bytes - returns a random byte string.
    • random_number - depending on the argument, returns either a randominteger in the range(0, n), or a random float between 0.0 and 1.0.
    • urlsafe_base64 - returns a random URL-safe Base64 encoded string.
    • uuid - return a version 4 random Universally Unique IDentifier.

There was a proposal to add a 'random.safe' submodule, quoting the Zenof Python 'Namespaces are one honking great idea' koan. However, theauthor of the Zen, Tim Peters, has come out against this idea [26], andrecommends a top-level module.

In discussion on the python-ideas mailing list so far, the name 'secrets'has received some approval, and no strong opposition.

There is already an existing third-party module with the same name [27],but it appears to be unused and abandoned.

  • Q: Is this a real problem? Surely MT is random enough that nobody canpredict its output.

    A: The consensus among security professionals is that MT is not safein security contexts. It is not difficult to reconstruct the internalstate of MT [28][29] and so predict all past and future values. Thereare a number of known, practical attacks on systems using MT forrandomness [30].

  • Q: Attacks on PHP are one thing, but are there any known attacks onPython software?

    A: Yes. There have been vulnerabilities in Zope and Plone at the veryleast. Hanno Schlichting commented [31]:

    Christian Heimes reported this issue to the Zope security team in 2012 [32],there are at least two related CVE vulnerabilities [33], and at least onework-around for this issue in Django [34].

  • Q: Is this an alternative to specialist cryptographic software such as SSL?

    A: No. This is a 'batteries included' solution, not a full-featured'nuclear reactor'. It is intended to mitigate against some basicsecurity errors, not be a solution to all security-related issues. Toquote Nick Coghlan referring to his earlier proposal [35]:

  • Q: What about a password generator?

    A: The consensus is that the requirements for password generators are toovariable for it to be a good match for the standard library [36]. No passwordgenerator will be included in the initial release of the module, instead itwill be given in the documentation as a recipe (à la the recipes in theitertools module) [37].

  • Q: Will secrets use /dev/random (which blocks) or /dev/urandom (whichdoesn't block) on Linux? What about other platforms?

    A: secrets will be based on os.urandom and random.SystemRandom,which are interfaces to your operating system's best source of cryptographicrandomness. On Linux, that may be /dev/urandom[38], on Windows it may beCryptGenRandom(), but see the documentation and/or source code for thedetailed implementation details.

[1]https://mail.python.org/pipermail/python-ideas/2015-September/035820.html
[2]https://docs.python.org/3/library/random.html
[3]As of the date of writing. Also, as Google search terms may beautomatically customised for the user without their knowledge, somereaders may see different results.
[4]http://interactivepython.org/runestone/static/everyday/2013/01/3_password.html
[5]http://stackoverflow.com/questions/3854692/generate-password-in-python
[6]http://stackoverflow.com/questions/3854692/generate-password-in-python/3854766#3854766
[7]https://mail.python.org/pipermail/python-ideas/2015-September/036238.html
[8]At least those who are motivated to read the source code and documentation.
[9]After considerable discussion, Guido ruled that the module need onlyprovide randbelow, and not similar functions randrange orrandint. http://code.activestate.com/lists/python-dev/138375/
[10]https://mail.python.org/pipermail/python-ideas/2015-September/036271.html
[11]https://mail.python.org/pipermail/python-ideas/2015-September/036350.html
[12]https://github.com/pyca/cryptography/issues/2347
[13]https://bitbucket.org/sdaprano/secrets
[14]
[15]https://mail.python.org/pipermail/python-ideas/2015-September/036474.html
[16]Link needed.
[17]By default PHP seeds the MT PRNG with the time (citation needed),which is exploitable by attackers, while Python seeds the PRNG withoutput from the system CSPRNG, which is believed to be much harder toexploit.

Leek: Government Secrets Mac Os Download

[18]http://legacy.python.org/dev/peps/pep-0504/
[19]http://php.net/manual/en/function.uniqid.php
[20]http://php.net/manual/en/function.openssl-random-pseudo-bytes.php
[21]Volunteers and patches are welcome.
[22]http://ifsec.blogspot.fr/2012/05/cross-domain-mathrandom-prediction.html
[23]https://developer.mozilla.org/en-US/docs/Web/API/RandomSource/getRandomValues
[24]https://nodejs.org/api/crypto.html
[25]http://ruby-doc.org/stdlib-2.1.2/libdoc/securerandom/rdoc/SecureRandom.html
Mac
[26]https://mail.python.org/pipermail/python-ideas/2015-September/036254.html
[27]https://pypi.python.org/pypi/secrets
[28]https://jazzy.id.au/2010/09/22/cracking_random_number_generators_part_3.html
[29]https://mail.python.org/pipermail/python-ideas/2015-September/036077.html
[30]https://media.blackhat.com/bh-us-12/Briefings/Argyros/BH_US_12_Argyros_PRNG_WP.pdf
[31]Personal communication, 2016-08-24.
[32]https://bugs.launchpad.net/zope2/+bug/1071067
[33]
[34]https://github.com/django/django/commit/1525874238fd705ec17a066291935a9316bd3044
[35]https://mail.python.org/pipermail/python-ideas/2015-September/036157.html
[36]
[37]https://mail.python.org/pipermail/python-ideas/2015-September/036488.html

Leek: Government Secrets Mac Os X

[38]

This document has been placed in the public domain.

Leek: Government Secrets Mac Os 11

Source: https://github.com/python/peps/blob/master/pep-0506.txt