Package test :: Module regrtest |
[hide private]
[frames] | no frames]

# Source Code for Module test.regrtest

       1  #! /usr/bin/python2.7
       2
       3  """
       4  Usage:
       5
       6  python -m test.regrtest [options] [test_name1 [test_name2 ...]]
       7  python path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]]
       8
       9
      10  If no arguments or options are provided, finds all files matching
      11  the pattern "test_*" in the Lib/test subdirectory and runs
      12  them in alphabetical order (but see -M and -u, below, for exceptions).
      13
      14  For more rigorous testing, it is useful to use the following
      15  command line:
      16
      17  python -E -tt -Wd -3 -m test.regrtest [options] [test_name1 ...]
      18
      19
      20  Options:
      21
      22  -h/--help       -- print this text and exit
      23
      24  Verbosity
      25
      26  -v/--verbose    -- run tests in verbose mode with output to stdout
      27  -w/--verbose2   -- re-run failed tests in verbose mode
      28  -W/--verbose3   -- re-run failed tests in verbose mode immediately
      29  -q/--quiet      -- no output unless one or more tests fail
      30  -S/--slow       -- print the slowest 10 tests
      31     --header     -- print header with interpreter info
      32
      33  Selecting tests
      34
      35  -r/--random     -- randomize test execution order (see below)
      36     --randseed   -- pass a random seed to reproduce a previous random run
      37  -f/--fromfile   -- read names of tests to run from a file (see below)
      38  -x/--exclude    -- arguments are tests to *exclude*
      39  -s/--single     -- single step through a set of tests (see below)
      40  -u/--use RES1,RES2,...
      41                  -- specify which special resource intensive tests to run
      42  -M/--memlimit LIMIT
      43                  -- run very large memory-consuming tests
      44
      45  Special runs
      46
      47  -l/--findleaks  -- if GC is available detect tests that leak memory
      48  -L/--runleaks   -- run the leaks(1) command just before exit
      49  -R/--huntrleaks RUNCOUNTS
      50                  -- search for reference leaks (needs debug build, v. slow)
      51  -j/--multiprocess PROCESSES
      52                  -- run PROCESSES processes at once
      53  -T/--coverage   -- turn on code coverage tracing using the trace module
      54  -D/--coverdir DIRECTORY
      55                  -- Directory where coverage files are put
      56  -N/--nocoverdir -- Put coverage files alongside modules
      57  -t/--threshold THRESHOLD
      58                  -- call gc.set_threshold(THRESHOLD)
      59  -F/--forever    -- run the specified tests in a loop, until an error happens
      60
      61
      62  Additional Option Details:
      63
      64  -r randomizes test execution order. You can use --randseed=int to provide a
      65  int seed value for the randomizer; this is useful for reproducing troublesome
      66  test orders.
      67
      68  -s On the first invocation of regrtest using -s, the first test file found
      69  or the first test file given on the command line is run, and the name of
      70  the next test is recorded in a file named pynexttest.  If run from the
      71  Python build directory, pynexttest is located in the 'build' subdirectory,
      72  otherwise it is located in tempfile.gettempdir().  On subsequent runs,
      73  the test in pynexttest is run, and the next test is written to pynexttest.
      74  When the last test has been run, pynexttest is deleted.  In this way it
      75  is possible to single step through the test files.  This is useful when
      76  doing memory analysis on the Python interpreter, which process tends to
      77  consume too many resources to run the full regression test non-stop.
      78
      79  -f reads the names of tests from the file given as f's argument, one
      80  or more test names per line.  Whitespace is ignored.  Blank lines and
      81  lines beginning with '#' are ignored.  This is especially useful for
      82  whittling down failures involving interactions among tests.
      83
      84  -L causes the leaks(1) command to be run just before exit if it exists.
      85  leaks(1) is available on Mac OS X and presumably on some other
      86  FreeBSD-derived systems.
      87
      88  -R runs each test several times and examines sys.gettotalrefcount() to
      89  see if the test appears to be leaking references.  The argument should
      90  be of the form stab:run:fname where 'stab' is the number of times the
      91  test is run to let gettotalrefcount settle down, 'run' is the number
      92  of times further it is run and 'fname' is the name of the file the
      93  reports are written to.  These parameters all have defaults (5, 4 and
      94  "reflog.txt" respectively), and the minimal invocation is '-R :'.
      95
      96  -M runs tests that require an exorbitant amount of memory. These tests
      97  typically try to ascertain containers keep working when containing more than
      98  2 billion objects, which only works on 64-bit systems. There are also some
      99  tests that try to exhaust the address space of the process, which only makes
     100  sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit,
     101  which is a string in the form of '2.5Gb', determines howmuch memory the
     102  tests will limit themselves to (but they may go slightly over.) The number
     103  shouldn't be more memory than the machine has (including swap memory). You
     104  should also keep in mind that swap memory is generally much, much slower
     105  than RAM, and setting memlimit to all available RAM or higher will heavily
     106  tax the machine. On the other hand, it is no use running these tests with a
     107  limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect
     108  to use more than memlimit memory will be skipped. The big-memory tests
     109  generally run very, very long.
     110
     111  -u is used to specify which special resource intensive tests to run,
     112  such as those requiring large file support or network connectivity.
     113  The argument is a comma-separated list of words indicating the
     114  resources to test.  Currently only the following are defined:
     115
     116      all -       Enable all special resources.
     117
     118      audio -     Tests that use the audio device.  (There are known
     119                  cases of broken audio drivers that can crash Python or
     120                  even the Linux kernel.)
     121
     122      curses -    Tests that use curses and will modify the terminal's
     123                  state and output modes.
     124
     125      largefile - It is okay to run some test that may create huge
     126                  files.  These tests can take a long time and may
     127                  consume >2GB of disk space temporarily.
     128
     129      network -   It is okay to run tests that use external network
     130                  resource, e.g. testing SSL support for sockets.
     131
     132      bsddb -     It is okay to run the bsddb testsuite, which takes
     133                  a long time to complete.
     134
     135      decimal -   Test the decimal module against a large suite that
     136                  verifies compliance with standards.
     137
     138      cpu -       Used for certain CPU-heavy tests.
     139
     140      subprocess  Run all tests for the subprocess module.
     141
     142      urlfetch -  It is okay to download files required on testing.
     143
     144      gui -       Run tests that require a running GUI.
     145
     146      xpickle -   Test pickle and cPickle against Python 2.4, 2.5 and 2.6 to
     147                  test backwards compatibility. These tests take a long time
     148                  to run.
     149
     150  To enable all resources except one, use '-uall,-<resource>'.  For
     151  example, to run all the tests except for the bsddb tests, give the
     152  option '-uall,-bsddb'.
     153  """
     154
     155  import StringIO
     156  import getopt
     157  import json
     158  import os
     159  import random
     160  import re
     161  import sys
     162  import time
     163  import traceback
     164  import warnings
     165  import unittest
     166  import tempfile
     167  import imp
     168  import platform
     169  import sysconfig
     170
     171
     172  # Some times __path__ and __file__ are not absolute (e.g. while running from
     173  # Lib/) and, if we change the CWD to run the tests in a temporary dir, some
     174  # imports might fail.  This affects only the modules imported before os.chdir().
     175  # These modules are searched first in sys.path[0] (so '' -- the CWD) and if
     176  # they are found in the CWD their __file__ and __path__ will be relative (this
     177  # happens before the chdir).  All the modules imported after the chdir, are
     178  # not found in the CWD, and since the other paths in sys.path[1:] are absolute
     179  # (site.py absolutize them), the __file__ and __path__ will be absolute too.
     180  # Therefore it is necessary to absolutize manually the __file__ and __path__ of
     181  # the packages to prevent later imports to fail when the CWD is different.
     182  for module in sys.modules.itervalues():
     183      if hasattr(module, '__path__'):
     184          module.__path__ = [os.path.abspath(path) for path in module.__path__]
     185      if hasattr(module, '__file__'):
     186          module.__file__ = os.path.abspath(module.__file__)
     187
     188
     189  # MacOSX (a.k.a. Darwin) has a default stack size that is too small
     190  # for deeply recursive regular expressions.  We see this as crashes in
     191  # the Python test suite when running test_re.py and test_sre.py.  The
     192  # fix is to set the stack limit to 2048.
     193  # This approach may also be useful for other Unixy platforms that
     194  # suffer from small default stack limits.
     195  if sys.platform == 'darwin':
     196      try:
     197          import resource
     198      except ImportError:
     199          pass
     200      else:
     201          soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
     202          newsoft = min(hard, max(soft, 1024*2048))
     203          resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))
     204
     205  # Test result constants.
     206  PASSED = 1
     207  FAILED = 0
     208  ENV_CHANGED = -1
     209  SKIPPED = -2
     210  RESOURCE_DENIED = -3
     211  INTERRUPTED = -4
     212
     213  from test import test_support
     214
     215  RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'bsddb',
     216                    'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui',
     217                    'xpickle')
     218
     219  TEMPDIR = os.path.abspath(tempfile.gettempdir())
     220
     221
     222 -def usage(code, msg=''):
     223      print __doc__
     224      if msg: print msg
     225      sys.exit(code)
     226
     227
     228 -def main(tests=None, testdir=None, verbose=0, quiet=False,
     229           exclude=False, single=False, randomize=False, fromfile=None,
     230           findleaks=False, use_resources=None, trace=False, coverdir='coverage',
     231           runleaks=False, huntrleaks=False, verbose2=False, print_slow=False,
     232           random_seed=None, use_mp=None, verbose3=False, forever=False,
     233           header=False):
     234      """Execute a test suite.
     235
     236      This also parses command-line options and modifies its behavior
     237      accordingly.
     238
     239      tests -- a list of strings containing test names (optional)
     240      testdir -- the directory in which to look for tests (optional)
     241
     242      Users other than the Python test suite will certainly want to
     243      specify testdir; if it's omitted, the directory containing the
     244      Python test suite is searched for.
     245
     246      If the tests argument is omitted, the tests listed on the
     247      command-line will be used.  If that's empty, too, then all *.py
     248      files beginning with test_ will be used.
     249
     250      The other default arguments (verbose, quiet, exclude,
     251      single, randomize, findleaks, use_resources, trace, coverdir,
     252      print_slow, and random_seed) allow programmers calling main()
     253      directly to set the values that would normally be set by flags
     254      on the command line.
     255      """
     256
     257      test_support.record_original_stdout(sys.stdout)
     258      try:
     259          opts, args = getopt.getopt(sys.argv[1:], 'hvqxsSrf:lu:t:TD:NLR:FwWM:j:',
     260              ['help', 'verbose', 'verbose2', 'verbose3', 'quiet',
     261               'exclude', 'single', 'slow', 'random', 'fromfile', 'findleaks',
     262               'use=', 'threshold=', 'trace', 'coverdir=', 'nocoverdir',
     263               'runleaks', 'huntrleaks=', 'memlimit=', 'randseed=',
     264               'multiprocess=', 'slaveargs=', 'forever', 'header'])
     265      except getopt.error, msg:
     266          usage(2, msg)
     267
     268      # Defaults
     269      if random_seed is None:
     270          random_seed = random.randrange(10000000)
     271      if use_resources is None:
     272          use_resources = []
     273      for o, a in opts:
     274          if o in ('-h', '--help'):
     275              usage(0)
     276          elif o in ('-v', '--verbose'):
     277              verbose += 1
     278          elif o in ('-w', '--verbose2'):
     279              verbose2 = True
     280          elif o in ('-W', '--verbose3'):
     281              verbose3 = True
     282          elif o in ('-q', '--quiet'):
     283              quiet = True;
     284              verbose = 0
     285          elif o in ('-x', '--exclude'):
     286              exclude = True
     287          elif o in ('-s', '--single'):
     288              single = True
     289          elif o in ('-S', '--slow'):
     290              print_slow = True
     291          elif o in ('-r', '--randomize'):
     292              randomize = True
     293          elif o == '--randseed':
     294              random_seed = int(a)
     295          elif o in ('-f', '--fromfile'):
     296              fromfile = a
     297          elif o in ('-l', '--findleaks'):
     298              findleaks = True
     299          elif o in ('-L', '--runleaks'):
     300              runleaks = True
     301          elif o in ('-t', '--threshold'):
     302              import gc
     303              gc.set_threshold(int(a))
     304          elif o in ('-T', '--coverage'):
     305              trace = True
     306          elif o in ('-D', '--coverdir'):
     307              coverdir = os.path.join(os.getcwd(), a)
     308          elif o in ('-N', '--nocoverdir'):
     309              coverdir = None
     310          elif o in ('-R', '--huntrleaks'):
     311              huntrleaks = a.split(':')
     312              if len(huntrleaks) not in (2, 3):
     313                  print a, huntrleaks
     314                  usage(2, '-R takes 2 or 3 colon-separated arguments')
     315              if not huntrleaks[0]:
     316                  huntrleaks[0] = 5
     317              else:
     318                  huntrleaks[0] = int(huntrleaks[0])
     319              if not huntrleaks[1]:
     320                  huntrleaks[1] = 4
     321              else:
     322                  huntrleaks[1] = int(huntrleaks[1])
     323              if len(huntrleaks) == 2 or not huntrleaks[2]:
     324                  huntrleaks[2:] = ["reflog.txt"]
     325          elif o in ('-M', '--memlimit'):
     326              test_support.set_memlimit(a)
     327          elif o in ('-u', '--use'):
     328              u = [x.lower() for x in a.split(',')]
     329              for r in u:
     330                  if r == 'all':
     331                      use_resources[:] = RESOURCE_NAMES
     332                      continue
     333                  remove = False
     334                  if r[0] == '-':
     335                      remove = True
     336                      r = r[1:]
     337                  if r not in RESOURCE_NAMES:
     338                      usage(1, 'Invalid -u/--use option: ' + a)
     339                  if remove:
     340                      if r in use_resources:
     341                          use_resources.remove(r)
     342                  elif r not in use_resources:
     343                      use_resources.append(r)
     344          elif o in ('-F', '--forever'):
     345              forever = True
     346          elif o in ('-j', '--multiprocess'):
     347              use_mp = int(a)
     348          elif o == '--header':
     349              header = True
     350          elif o == '--slaveargs':
     351              args, kwargs = json.loads(a)
     352              try:
     353                  result = runtest(*args, **kwargs)
     354              except BaseException, e:
     355                  result = INTERRUPTED, e.__class__.__name__
     356              print   # Force a newline (just in case)
     357              print json.dumps(result)
     358              sys.exit(0)
     359          else:
     360              print >>sys.stderr, ("No handler for option {}.  Please "
     361                  "report this as a bug at http://bugs.python.org.").format(o)
     362              sys.exit(1)
     363      if single and fromfile:
     364          usage(2, "-s and -f don't go together!")
     365      if use_mp and trace:
     366          usage(2, "-T and -j don't go together!")
     367      if use_mp and findleaks:
     368          usage(2, "-l and -j don't go together!")
     369
     370      good = []
     371      bad = []
     372      skipped = []
     373      resource_denieds = []
     374      environment_changed = []
     375      interrupted = False
     376
     377      if findleaks:
     378          try:
     379              import gc
     380          except ImportError:
     381              print 'No GC available, disabling findleaks.'
     382              findleaks = False
     383          else:
     384              # Uncomment the line below to report garbage that is not
     385              # freeable by reference counting alone.  By default only
     386              # garbage that is not collectable by the GC is reported.
     387              #gc.set_debug(gc.DEBUG_SAVEALL)
     388              found_garbage = []
     389
     390      if single:
     391          filename = os.path.join(TEMPDIR, 'pynexttest')
     392          try:
     393              fp = open(filename, 'r')
     394              next_test = fp.read().strip()
     395              tests = [next_test]
     396              fp.close()
     397          except IOError:
     398              pass
     399
     400      if fromfile:
     401          tests = []
     402          fp = open(os.path.join(test_support.SAVEDCWD, fromfile))
     403          for line in fp:
     404              guts = line.split() # assuming no test has whitespace in its name
     405              if guts and not guts[0].startswith('#'):
     406                  tests.extend(guts)
     407          fp.close()
     408
     409      # Strip .py extensions.
     410      removepy(args)
     411      removepy(tests)
     412
     413      stdtests = STDTESTS[:]
     414      nottests = NOTTESTS.copy()
     415      if exclude:
     416          for arg in args:
     417              if arg in stdtests:
     418                  stdtests.remove(arg)
     419              nottests.add(arg)
     420          args = []
     421
     422      # For a partial run, we do not need to clutter the output.
     423      if verbose or header or not (quiet or single or tests or args):
     424          # Print basic platform information
     425          print "==", platform.python_implementation(), \
     426                      " ".join(sys.version.split())
     427          print "==  ", platform.platform(aliased=True), \
     428                        "%s-endian" % sys.byteorder
     429          print "==  ", os.getcwd()
     430          print "Testing with flags:", sys.flags
     431
     432      alltests = findtests(testdir, stdtests, nottests)
     433      selected = tests or args or alltests
     434      if single:
     435          selected = selected[:1]
     436          try:
     437              next_single_test = alltests[alltests.index(selected[0])+1]
     438          except IndexError:
     439              next_single_test = None
     440      if randomize:
     441          random.seed(random_seed)
     442          print "Using random seed", random_seed
     443          random.shuffle(selected)
     444      if trace:
     445          import trace
     446          tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix],
     447                               trace=False, count=True)
     448
     449      test_times = []
     450      test_support.use_resources = use_resources
     451      save_modules = sys.modules.keys()
     452
     453      def accumulate_result(test, result):
     454          ok, test_time = result
     455          test_times.append((test_time, test))
     456          if ok == PASSED:
     457              good.append(test)
     458          elif ok == FAILED:
     459              bad.append(test)
     460          elif ok == ENV_CHANGED:
     461              bad.append(test)
     462              environment_changed.append(test)
     463          elif ok == SKIPPED:
     464              skipped.append(test)
     465          elif ok == RESOURCE_DENIED:
     466              skipped.append(test)
     467              resource_denieds.append(test)
     468
     469      if forever:
     470          def test_forever(tests=list(selected)):
     471              while True:
     472                  for test in tests:
     473                      yield test
     474                      if bad:
     475                          return
     476          tests = test_forever()
     477      else:
     478          tests = iter(selected)
     479
     480      if use_mp:
     481          try:
     482              from threading import Thread
     483          except ImportError:
     484              print "Multiprocess option requires thread support"
     485              sys.exit(2)
     486          from Queue import Queue
     487          from subprocess import Popen, PIPE
     488          debug_output_pat = re.compile(r"\[\d+ refs\]$")
     489          output = Queue()
     490          def tests_and_args():
     491              for test in tests:
     492                  args_tuple = (
     493                      (test, verbose, quiet),
     494                      dict(huntrleaks=huntrleaks, use_resources=use_resources)
     495                  )
     496                  yield (test, args_tuple)
     497          pending = tests_and_args()
     498          opt_args = test_support.args_from_interpreter_flags()
     499          base_cmd = [sys.executable] + opt_args + ['-m', 'test.regrtest']
     500          def work():
     501              # A worker thread.
     502              try:
     503                  while True:
     504                      try:
     505                          test, args_tuple = next(pending)
     506                      except StopIteration:
     507                          output.put((None, None, None, None))
     508                          return
     509                      # -E is needed by some tests, e.g. test_import
     510                      popen = Popen(base_cmd + ['--slaveargs', json.dumps(args_tuple)],
     511                                     stdout=PIPE, stderr=PIPE,
     512                                     universal_newlines=True,
     513                                     close_fds=(os.name != 'nt'))
     514                      stdout, stderr = popen.communicate()
     515                      # Strip last refcount output line if it exists, since it
     516                      # comes from the shutdown of the interpreter in the subcommand.
     517                      stderr = debug_output_pat.sub("", stderr)
     518                      stdout, _, result = stdout.strip().rpartition("\n")
     519                      if not result:
     520                          output.put((None, None, None, None))
     521                          return
     522                      result = json.loads(result)
     523                      if not quiet:
     524                          stdout = test+'\n'+stdout
     525                      output.put((test, stdout.rstrip(), stderr.rstrip(), result))
     526              except BaseException:
     527                  output.put((None, None, None, None))
     528                  raise
     529          workers = [Thread(target=work) for i in range(use_mp)]
     530          for worker in workers:
     531              worker.start()
     532          finished = 0
     533          try:
     534              while finished < use_mp:
     535                  test, stdout, stderr, result = output.get()
     536                  if test is None:
     537                      finished += 1
     538                      continue
     539                  if stdout:
     540                      print stdout
     541                  if stderr:
     542                      print >>sys.stderr, stderr
     543                  if result[0] == INTERRUPTED:
     544                      assert result[1] == 'KeyboardInterrupt'
     545                      raise KeyboardInterrupt   # What else?
     546                  accumulate_result(test, result)
     547          except KeyboardInterrupt:
     548              interrupted = True
     549              pending.close()
     550          for worker in workers:
     551              worker.join()
     552      else:
     553          for test in tests:
     554              if not quiet:
     555                  print test
     556                  sys.stdout.flush()
     557              if trace:
     558                  # If we're tracing code coverage, then we don't exit with status
     559                  # if on a false return value from main.
     560                  tracer.runctx('runtest(test, verbose, quiet)',
     561                                globals=globals(), locals=vars())
     562              else:
     563                  try:
     564                      result = runtest(test, verbose, quiet, huntrleaks)
     565                      accumulate_result(test, result)
     566                      if verbose3 and result[0] == FAILED:
     567                          print "Re-running test %r in verbose mode" % test
     568                          runtest(test, True, quiet, huntrleaks)
     569                  except KeyboardInterrupt:
     570                      interrupted = True
     571                      break
     572                  except:
     573                      raise
     574              if findleaks:
     575                  gc.collect()
     576                  if gc.garbage:
     577                      print "Warning: test created", len(gc.garbage),
     578                      print "uncollectable object(s)."
     579                      # move the uncollectable objects somewhere so we don't see
     580                      # them again
     581                      found_garbage.extend(gc.garbage)
     582                      del gc.garbage[:]
     583              # Unload the newly imported modules (best effort finalization)
     584              for module in sys.modules.keys():
     585                  if module not in save_modules and module.startswith("test."):
     586                      test_support.unload(module)
     587
     588      if interrupted:
     589          # print a newline after ^C
     590          print
     591          print "Test suite interrupted by signal SIGINT."
     592          omitted = set(selected) - set(good) - set(bad) - set(skipped)
     593          print count(len(omitted), "test"), "omitted:"
     594          printlist(omitted)
     595      if good and not quiet:
     596          if not bad and not skipped and not interrupted and len(good) > 1:
     597              print "All",
     598          print count(len(good), "test"), "OK."
     599      if print_slow:
     600          test_times.sort(reverse=True)
     601          print "10 slowest tests:"
     602          for time, test in test_times[:10]:
     603              print "%s: %.1fs" % (test, time)
     604      if bad:
     605          bad = set(bad) - set(environment_changed)
     606          if bad:
     607              print count(len(bad), "test"), "failed:"
     608              printlist(bad)
     609          if environment_changed:
     610              print "{} altered the execution environment:".format(
     611                  count(len(environment_changed), "test"))
     612              printlist(environment_changed)
     613      if skipped and not quiet:
     614          print count(len(skipped), "test"), "skipped:"
     615          printlist(skipped)
     616
     617          e = _ExpectedSkips()
     618          plat = sys.platform
     619          if e.isvalid():
     620              surprise = set(skipped) - e.getexpected() - set(resource_denieds)
     621              if surprise:
     622                  print count(len(surprise), "skip"), \
     623                        "unexpected on", plat + ":"
     624                  printlist(surprise)
     625              else:
     626                  print "Those skips are all expected on", plat + "."
     627          else:
     628              print "Ask someone to teach regrtest.py about which tests are"
     629              print "expected to get skipped on", plat + "."
     630
     631      if verbose2 and bad:
     632          print "Re-running failed tests in verbose mode"
     633          for test in bad:
     634              print "Re-running test %r in verbose mode" % test
     635              sys.stdout.flush()
     636              try:
     637                  test_support.verbose = True
     638                  ok = runtest(test, True, quiet, huntrleaks)
     639              except KeyboardInterrupt:
     640                  # print a newline separate from the ^C
     641                  print
     642                  break
     643              except:
     644                  raise
     645
     646      if single:
     647          if next_single_test:
     648              with open(filename, 'w') as fp:
     649                  fp.write(next_single_test + '\n')
     650          else:
     651              os.unlink(filename)
     652
     653      if trace:
     654          r = tracer.results()
     655          r.write_results(show_missing=True, summary=True, coverdir=coverdir)
     656
     657      if runleaks:
     658          os.system("leaks %d" % os.getpid())
     659
     660      sys.exit(len(bad) > 0 or interrupted)
     661
     662
     663  STDTESTS = [
     664      'test_grammar',
     665      'test_opcodes',
     666      'test_dict',
     667      'test_builtin',
     668      'test_exceptions',
     669      'test_types',
     670      'test_unittest',
     671      'test_doctest',
     672      'test_doctest2',
     673  ]
     674
     675  NOTTESTS = {
     676      'test_support',
     677      'test_future1',
     678      'test_future2',
     679  }
     680
     681 -def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS):
     682      """Return a list of all applicable test modules."""
     683      testdir = findtestdir(testdir)
     684      names = os.listdir(testdir)
     685      tests = []
     686      others = set(stdtests) | nottests
     687      for name in names:
     688          modname, ext = os.path.splitext(name)
     689          if modname[:5] == "test_" and ext == ".py" and modname not in others:
     690              tests.append(modname)
     691      return stdtests + sorted(tests)
     692
     693 -def runtest(test, verbose, quiet,
     694              huntrleaks=False, use_resources=None):
     695      """Run a single test.
     696
     697      test -- the name of the test
     698      verbose -- if true, print more messages
     699      quiet -- if true, don't print 'skipped' messages (probably redundant)
     700      test_times -- a list of (time, test_name) pairs
     701      huntrleaks -- run multiple times to test for leaks; requires a debug
     702                    build; a triple corresponding to -R's three arguments
     703      Returns one of the test result constants:
     704          INTERRUPTED      KeyboardInterrupt when run under -j
     705          RESOURCE_DENIED  test skipped because resource denied
     706          SKIPPED          test skipped for some other reason
     707          ENV_CHANGED      test failed because it changed the execution environment
     708          FAILED           test failed
     709          PASSED           test passed
     710      """
     711
     712      test_support.verbose = verbose  # Tell tests to be moderately quiet
     713      if use_resources is not None:
     714          test_support.use_resources = use_resources
     715      try:
     716          return runtest_inner(test, verbose, quiet, huntrleaks)
     717      finally:
     718          cleanup_test_droppings(test, verbose)
     719
     720
     721  # Unit tests are supposed to leave the execution environment unchanged
     722  # once they complete.  But sometimes tests have bugs, especially when
     723  # tests fail, and the changes to environment go on to mess up other
     724  # tests.  This can cause issues with buildbot stability, since tests
     725  # are run in random order and so problems may appear to come and go.
     726  # There are a few things we can save and restore to mitigate this, and
     727  # the following context manager handles this task.
     728
     729 -class saved_test_environment:
     730      """Save bits of the test environment and restore them at block exit.
     731
     732          with saved_test_environment(testname, verbose, quiet):
     733              #stuff
     734
     735      Unless quiet is True, a warning is printed to stderr if any of
     736      the saved items was changed by the test.  The attribute 'changed'
     737      is initially False, but is set to True if a change is detected.
     738
     739      If verbose is more than 1, the before and after state of changed
     740      items is also printed.
     741      """
     742
     743      changed = False
     744
     745 -    def __init__(self, testname, verbose=0, quiet=False):
     746          self.testname = testname
     747          self.verbose = verbose
     748          self.quiet = quiet
     749
     750      # To add things to save and restore, add a name XXX to the resources list
     751      # and add corresponding get_XXX/restore_XXX functions.  get_XXX should
     752      # return the value to be saved and compared against a second call to the
     753      # get function when test execution completes.  restore_XXX should accept
     754      # the saved value and restore the resource using it.  It will be called if
     755      # and only if a change in the value is detected.
     756      #
     757      # Note: XXX will have any '.' replaced with '_' characters when determining
     758      # the corresponding method names.
     759
     760      resources = ('sys.argv', 'cwd', 'sys.stdin', 'sys.stdout', 'sys.stderr',
     761                   'os.environ', 'sys.path', 'asyncore.socket_map')
     762
     763 -    def get_sys_argv(self):
     764          return id(sys.argv), sys.argv, sys.argv[:]
     765 -    def restore_sys_argv(self, saved_argv):
     766          sys.argv = saved_argv[1]
     767          sys.argv[:] = saved_argv[2]
     768
     769 -    def get_cwd(self):
     770          return os.getcwd()
     771 -    def restore_cwd(self, saved_cwd):
     772          os.chdir(saved_cwd)
     773
     774 -    def get_sys_stdout(self):
     775          return sys.stdout
     776 -    def restore_sys_stdout(self, saved_stdout):
     777          sys.stdout = saved_stdout
     778
     779 -    def get_sys_stderr(self):
     780          return sys.stderr
     781 -    def restore_sys_stderr(self, saved_stderr):
     782          sys.stderr = saved_stderr
     783
     784 -    def get_sys_stdin(self):
     785          return sys.stdin
     786 -    def restore_sys_stdin(self, saved_stdin):
     787          sys.stdin = saved_stdin
     788
     789 -    def get_os_environ(self):
     790          return id(os.environ), os.environ, dict(os.environ)
     791 -    def restore_os_environ(self, saved_environ):
     792          os.environ = saved_environ[1]
     793          os.environ.clear()
     794          os.environ.update(saved_environ[2])
     795
     796 -    def get_sys_path(self):
     797          return id(sys.path), sys.path, sys.path[:]
     798 -    def restore_sys_path(self, saved_path):
     799          sys.path = saved_path[1]
     800          sys.path[:] = saved_path[2]
     801
     802 -    def get_asyncore_socket_map(self):
     803          asyncore = sys.modules.get('asyncore')
     804          # XXX Making a copy keeps objects alive until __exit__ gets called.
     805          return asyncore and asyncore.socket_map.copy() or {}
     806 -    def restore_asyncore_socket_map(self, saved_map):
     807          asyncore = sys.modules.get('asyncore')
     808          if asyncore is not None:
     809              asyncore.close_all(ignore_all=True)
     810              asyncore.socket_map.update(saved_map)
     811
     812 -    def resource_info(self):
     813          for name in self.resources:
     814              method_suffix = name.replace('.', '_')
     815              get_name = 'get_' + method_suffix
     816              restore_name = 'restore_' + method_suffix
     817              yield name, getattr(self, get_name), getattr(self, restore_name)
     818
     819 -    def __enter__(self):
     820          self.saved_values = dict((name, get()) for name, get, restore
     821                                                     in self.resource_info())
     822          return self
     823
     824 -    def __exit__(self, exc_type, exc_val, exc_tb):
     825          saved_values = self.saved_values
     826          del self.saved_values
     827          for name, get, restore in self.resource_info():
     828              current = get()
     829              original = saved_values.pop(name)
     830              # Check for changes to the resource's value
     831              if current != original:
     832                  self.changed = True
     833                  restore(original)
     834                  if not self.quiet:
     835                      print >>sys.stderr, (
     836                            "Warning -- {} was modified by {}".format(
     837                                                   name, self.testname))
     838                      if self.verbose > 1:
     839                          print >>sys.stderr, (
     840                                "  Before: {}\n  After:  {} ".format(
     841                                                    original, current))
     842              # XXX (ncoghlan): for most resources (e.g. sys.path) identity
     843              # matters at least as much as value. For others (e.g. cwd),
     844              # identity is irrelevant. Should we add a mechanism to check
     845              # for substitution in the cases where it matters?
     846          return False
     847
     848
     849 -def runtest_inner(test, verbose, quiet, huntrleaks=False):
     850      test_support.unload(test)
     851      if verbose:
     852          capture_stdout = None
     853      else:
     854          capture_stdout = StringIO.StringIO()
     855
     856      test_time = 0.0
     857      refleak = False  # True if the test leaked references.
     858      try:
     859          save_stdout = sys.stdout
     860          try:
     861              if capture_stdout:
     862                  sys.stdout = capture_stdout
     863              if test.startswith('test.'):
     864                  abstest = test
     865              else:
     866                  # Always import it from the test package
     867                  abstest = 'test.' + test
     868              with saved_test_environment(test, verbose, quiet) as environment:
     869                  start_time = time.time()
     870                  the_package = __import__(abstest, globals(), locals(), [])
     871                  the_module = getattr(the_package, test)
     872                  # Old tests run to completion simply as a side-effect of
     873                  # being imported.  For tests based on unittest or doctest,
     874                  # explicitly invoke their test_main() function (if it exists).
     875                  indirect_test = getattr(the_module, "test_main", None)
     876                  if indirect_test is not None:
     877                      indirect_test()
     878                  if huntrleaks:
     879                      refleak = dash_R(the_module, test, indirect_test,
     880                          huntrleaks)
     881                  test_time = time.time() - start_time
     882          finally:
     883              sys.stdout = save_stdout
     884      except test_support.ResourceDenied, msg:
     885          if not quiet:
     886              print test, "skipped --", msg
     887              sys.stdout.flush()
     888          return RESOURCE_DENIED, test_time
     889      except unittest.SkipTest, msg:
     890          if not quiet:
     891              print test, "skipped --", msg
     892              sys.stdout.flush()
     893          return SKIPPED, test_time
     894      except KeyboardInterrupt:
     895          raise
     896      except test_support.TestFailed, msg:
     897          print >>sys.stderr, "test", test, "failed --", msg
     898          sys.stderr.flush()
     899          return FAILED, test_time
     900      except:
     901          type, value = sys.exc_info()[:2]
     902          print >>sys.stderr, "test", test, "crashed --", str(type) + ":", value
     903          sys.stderr.flush()
     904          if verbose:
     905              traceback.print_exc(file=sys.stderr)
     906              sys.stderr.flush()
     907          return FAILED, test_time
     908      else:
     909          if refleak:
     910              return FAILED, test_time
     911          if environment.changed:
     912              return ENV_CHANGED, test_time
     913          # Except in verbose mode, tests should not print anything
     914          if verbose or huntrleaks:
     915              return PASSED, test_time
     916          output = capture_stdout.getvalue()
     917          if not output:
     918              return PASSED, test_time
     919          print "test", test, "produced unexpected output:"
     920          print "*" * 70
     921          print output
     922          print "*" * 70
     923          sys.stdout.flush()
     924          return FAILED, test_time
     925
     926 -def cleanup_test_droppings(testname, verbose):
     927      import shutil
     928      import stat
     929      import gc
     930
     931      # First kill any dangling references to open files etc.
     932      gc.collect()
     933
     934      # Try to clean up junk commonly left behind.  While tests shouldn't leave
     935      # any files or directories behind, when a test fails that can be tedious
     936      # for it to arrange.  The consequences can be especially nasty on Windows,
     937      # since if a test leaves a file open, it cannot be deleted by name (while
     938      # there's nothing we can do about that here either, we can display the
     939      # name of the offending test, which is a real help).
     940      for name in (test_support.TESTFN,
     941                   "db_home",
     942                  ):
     943          if not os.path.exists(name):
     944              continue
     945
     946          if os.path.isdir(name):
     947              kind, nuker = "directory", shutil.rmtree
     948          elif os.path.isfile(name):
     949              kind, nuker = "file", os.unlink
     950          else:
     951              raise SystemError("os.path says %r exists but is neither "
     952                                "directory nor file" % name)
     953
     954          if verbose:
     955              print "%r left behind %s %r" % (testname, kind, name)
     956          try:
     957              # if we have chmod, fix possible permissions problems
     958              # that might prevent cleanup
     959              if (hasattr(os, 'chmod')):
     960                  os.chmod(name, stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO)
     961              nuker(name)
     962          except Exception, msg:
     963              print >> sys.stderr, ("%r left behind %s %r and it couldn't be "
     964                  "removed: %s" % (testname, kind, name, msg))
     965
     966 -def dash_R(the_module, test, indirect_test, huntrleaks):
     967      """Run a test multiple times, looking for reference leaks.
     968
     969      Returns:
     970          False if the test didn't leak references; True if we detected refleaks.
     971      """
     972      # This code is hackish and inelegant, but it seems to do the job.
     973      import copy_reg, _abcoll, _pyio
     974
     975      if not hasattr(sys, 'gettotalrefcount'):
     976          raise Exception("Tracking reference leaks requires a debug build "
     977                          "of Python")
     978
     979      # Save current values for dash_R_cleanup() to restore.
     980      fs = warnings.filters[:]
     981      ps = copy_reg.dispatch_table.copy()
     982      pic = sys.path_importer_cache.copy()
     983      try:
     984          import zipimport
     985      except ImportError:
     986          zdc = None # Run unmodified on platforms without zipimport support
     987      else:
     988          zdc = zipimport._zip_directory_cache.copy()
     989      abcs = {}
     990      modules = _abcoll, _pyio
     991      for abc in [getattr(mod, a) for mod in modules for a in mod.__all__]:
     992          # XXX isinstance(abc, ABCMeta) leads to infinite recursion
     993          if not hasattr(abc, '_abc_registry'):
     994              continue
     995          for obj in abc.__subclases__() + [abc]:
     996              abcs[obj] = obj._abc_registry.copy()
     997
     998      if indirect_test:
     999          def run_the_test():
    1000              indirect_test()
    1001      else:
    1002          def run_the_test():
    1003              imp.reload(the_module)
    1004
    1005      deltas = []
    1006      nwarmup, ntracked, fname = huntrleaks
    1007      fname = os.path.join(test_support.SAVEDCWD, fname)
    1008      repcount = nwarmup + ntracked
    1009      print >> sys.stderr, "beginning", repcount, "repetitions"
    1010      print >> sys.stderr, ("1234567890"*(repcount//10 + 1))[:repcount]
    1011      dash_R_cleanup(fs, ps, pic, zdc, abcs)
    1012      for i in range(repcount):
    1013          rc_before = sys.gettotalrefcount()
    1014          run_the_test()
    1015          sys.stderr.write('.')
    1016          dash_R_cleanup(fs, ps, pic, zdc, abcs)
    1017          rc_after = sys.gettotalrefcount()
    1018          if i >= nwarmup:
    1019              deltas.append(rc_after - rc_before)
    1020      print >> sys.stderr
    1021      if any(deltas):
    1022          msg = '%s leaked %s references, sum=%s' % (test, deltas, sum(deltas))
    1023          print >> sys.stderr, msg
    1024          with open(fname, "a") as refrep:
    1025              print >> refrep, msg
    1026              refrep.flush()
    1027          return True
    1028      return False
    1029
    1030 -def dash_R_cleanup(fs, ps, pic, zdc, abcs):
    1031      import gc, copy_reg
    1032      import _strptime, linecache
    1033      dircache = test_support.import_module('dircache', deprecated=True)
    1034      import urlparse, urllib, urllib2, mimetypes, doctest
    1035      import struct, filecmp
    1036      from distutils.dir_util import _path_created
    1037
    1038      # Clear the warnings registry, so they can be displayed again
    1039      for mod in sys.modules.values():
    1040          if hasattr(mod, '__warningregistry__'):
    1041              del mod.__warningregistry__
    1042
    1043      # Restore some original values.
    1044      warnings.filters[:] = fs
    1045      copy_reg.dispatch_table.clear()
    1046      copy_reg.dispatch_table.update(ps)
    1047      sys.path_importer_cache.clear()
    1048      sys.path_importer_cache.update(pic)
    1049      try:
    1050          import zipimport
    1051      except ImportError:
    1052          pass # Run unmodified on platforms without zipimport support
    1053      else:
    1054          zipimport._zip_directory_cache.clear()
    1055          zipimport._zip_directory_cache.update(zdc)
    1056
    1057      # clear type cache
    1058      sys._clear_type_cache()
    1059
    1060      # Clear ABC registries, restoring previously saved ABC registries.
    1061      for abc, registry in abcs.items():
    1062          abc._abc_registry = registry.copy()
    1063          abc._abc_cache.clear()
    1064          abc._abc_negative_cache.clear()
    1065
    1066      # Clear assorted module caches.
    1067      _path_created.clear()
    1068      re.purge()
    1069      _strptime._regex_cache.clear()
    1070      urlparse.clear_cache()
    1071      urllib.urlcleanup()
    1072      urllib2.install_opener(None)
    1073      dircache.reset()
    1074      linecache.clearcache()
    1075      mimetypes._default_mime_types()
    1076      filecmp._cache.clear()
    1077      struct._clearcache()
    1078      doctest.master = None
    1079
    1080      # Collect cyclic trash.
    1081      gc.collect()
    1082
    1083 -def findtestdir(path=None):
    1084      return path or os.path.dirname(__file__) or os.curdir
    1085
    1086 -def removepy(names):
    1087      if not names:
    1088          return
    1089      for idx, name in enumerate(names):
    1090          basename, ext = os.path.splitext(name)
    1091          if ext == '.py':
    1092              names[idx] = basename
    1093
    1094 -def count(n, word):
    1095      if n == 1:
    1096          return "%d %s" % (n, word)
    1097      else:
    1098          return "%d %ss" % (n, word)
    1099
    1100 -def printlist(x, width=70, indent=4):
    1101      """Print the elements of iterable x to stdout.
    1102
    1103      Optional arg width (default 70) is the maximum line length.
    1104      Optional arg indent (default 4) is the number of blanks with which to
    1105      begin each line.
    1106      """
    1107
    1108      from textwrap import fill
    1109      blanks = ' ' * indent
    1110      # Print the sorted list: 'x' may be a '--random' list or a set()
    1111      print fill(' '.join(str(elt) for elt in sorted(x)), width,
    1112                 initial_indent=blanks, subsequent_indent=blanks)
    1113
    1114  # Map sys.platform to a string containing the basenames of tests
    1115  # expected to be skipped on that platform.
    1116  #
    1117  # Special cases:
    1118  #     test_pep277
    1119  #         The _ExpectedSkips constructor adds this to the set of expected
    1120  #         skips if not os.path.supports_unicode_filenames.
    1121  #     test_timeout
    1122  #         Controlled by test_timeout.skip_expected.  Requires the network
    1123  #         resource and a socket module.
    1124  #
    1125  # Tests that are expected to be skipped everywhere except on one platform
    1126  # are also handled separately.
    1127
    1128  _expectations = {
    1129      'win32':
    1130          """
    1131          test__locale
    1132          test_bsddb185
    1133          test_bsddb3
    1134          test_commands
    1135          test_crypt
    1136          test_curses
    1137          test_dbm
    1138          test_dl
    1139          test_fcntl
    1140          test_fork1
    1141          test_epoll
    1142          test_gdbm
    1143          test_grp
    1144          test_ioctl
    1145          test_largefile
    1146          test_kqueue
    1147          test_mhlib
    1148          test_openpty
    1149          test_ossaudiodev
    1150          test_pipes
    1151          test_poll
    1152          test_posix
    1153          test_pty
    1154          test_pwd
    1155          test_resource
    1156          test_signal
    1157          test_threadsignals
    1158          test_timing
    1159          test_wait3
    1160          test_wait4
    1161          """,
    1162      'linux2':
    1163          """
    1164          test_bsddb185
    1165          test_curses
    1166          test_dl
    1167          test_largefile
    1168          test_kqueue
    1169          test_ossaudiodev
    1170          """,
    1171      'unixware7':
    1172          """
    1173          test_bsddb
    1174          test_bsddb185
    1175          test_dl
    1176          test_epoll
    1177          test_largefile
    1178          test_kqueue
    1179          test_minidom
    1180          test_openpty
    1181          test_pyexpat
    1182          test_sax
    1183          test_sundry
    1184          """,
    1185      'openunix8':
    1186          """
    1187          test_bsddb
    1188          test_bsddb185
    1189          test_dl
    1190          test_epoll
    1191          test_largefile
    1192          test_kqueue
    1193          test_minidom
    1194          test_openpty
    1195          test_pyexpat
    1196          test_sax
    1197          test_sundry
    1198          """,
    1199      'sco_sv3':
    1200          """
    1201          test_asynchat
    1202          test_bsddb
    1203          test_bsddb185
    1204          test_dl
    1205          test_fork1
    1206          test_epoll
    1207          test_gettext
    1208          test_largefile
    1209          test_locale
    1210          test_kqueue
    1211          test_minidom
    1212          test_openpty
    1213          test_pyexpat
    1214          test_queue
    1215          test_sax
    1216          test_sundry
    1217          test_thread
    1218          test_threaded_import
    1219          test_threadedtempfile
    1220          test_threading
    1221          """,
    1222      'riscos':
    1223          """
    1224          test_asynchat
    1225          test_atexit
    1226          test_bsddb
    1227          test_bsddb185
    1228          test_bsddb3
    1229          test_commands
    1230          test_crypt
    1231          test_dbm
    1232          test_dl
    1233          test_fcntl
    1234          test_fork1
    1235          test_epoll
    1236          test_gdbm
    1237          test_grp
    1238          test_largefile
    1239          test_locale
    1240          test_kqueue
    1241          test_mmap
    1242          test_openpty
    1243          test_poll
    1244          test_popen2
    1245          test_pty
    1246          test_pwd
    1247          test_strop
    1248          test_sundry
    1249          test_thread
    1250          test_threaded_import
    1251          test_threadedtempfile
    1252          test_threading
    1253          test_timing
    1254          """,
    1255      'darwin':
    1256          """
    1257          test__locale
    1258          test_bsddb
    1259          test_bsddb3
    1260          test_curses
    1261          test_epoll
    1262          test_gdb
    1263          test_gdbm
    1264          test_largefile
    1265          test_locale
    1266          test_kqueue
    1267          test_minidom
    1268          test_ossaudiodev
    1269          test_poll
    1270          """,
    1271      'sunos5':
    1272          """
    1273          test_bsddb
    1274          test_bsddb185
    1275          test_curses
    1276          test_dbm
    1277          test_epoll
    1278          test_kqueue
    1279          test_gdbm
    1280          test_gzip
    1281          test_openpty
    1282          test_zipfile
    1283          test_zlib
    1284          """,
    1285      'hp-ux11':
    1286          """
    1287          test_bsddb
    1288          test_bsddb185
    1289          test_curses
    1290          test_dl
    1291          test_epoll
    1292          test_gdbm
    1293          test_gzip
    1294          test_largefile
    1295          test_locale
    1296          test_kqueue
    1297          test_minidom
    1298          test_openpty
    1299          test_pyexpat
    1300          test_sax
    1301          test_zipfile
    1302          test_zlib
    1303          """,
    1304      'atheos':
    1305          """
    1306          test_bsddb185
    1307          test_curses
    1308          test_dl
    1309          test_gdbm
    1310          test_epoll
    1311          test_largefile
    1312          test_locale
    1313          test_kqueue
    1314          test_mhlib
    1315          test_mmap
    1316          test_poll
    1317          test_popen2
    1318          test_resource
    1319          """,
    1320      'cygwin':
    1321          """
    1322          test_bsddb185
    1323          test_bsddb3
    1324          test_curses
    1325          test_dbm
    1326          test_epoll
    1327          test_ioctl
    1328          test_kqueue
    1329          test_largefile
    1330          test_locale
    1331          test_ossaudiodev
    1332          test_socketserver
    1333          """,
    1334      'os2emx':
    1335          """
    1336          test_audioop
    1337          test_bsddb185
    1338          test_bsddb3
    1339          test_commands
    1340          test_curses
    1341          test_dl
    1342          test_epoll
    1343          test_kqueue
    1344          test_largefile
    1345          test_mhlib
    1346          test_mmap
    1347          test_openpty
    1348          test_ossaudiodev
    1349          test_pty
    1350          test_resource
    1351          test_signal
    1352          """,
    1353      'freebsd4':
    1354          """
    1355          test_bsddb
    1356          test_bsddb3
    1357          test_epoll
    1358          test_gdbm
    1359          test_locale
    1360          test_ossaudiodev
    1361          test_pep277
    1362          test_pty
    1363          test_socketserver
    1364          test_tcl
    1365          test_tk
    1366          test_ttk_guionly
    1367          test_ttk_textonly
    1368          test_timeout
    1369          test_urllibnet
    1370          test_multiprocessing
    1371          """,
    1372      'aix5':
    1373          """
    1374          test_bsddb
    1375          test_bsddb185
    1376          test_bsddb3
    1377          test_bz2
    1378          test_dl
    1379          test_epoll
    1380          test_gdbm
    1381          test_gzip
    1382          test_kqueue
    1383          test_ossaudiodev
    1384          test_tcl
    1385          test_tk
    1386          test_ttk_guionly
    1387          test_ttk_textonly
    1388          test_zipimport
    1389          test_zlib
    1390          """,
    1391      'openbsd3':
    1392          """
    1393          test_ascii_formatd
    1394          test_bsddb
    1395          test_bsddb3
    1396          test_ctypes
    1397          test_dl
    1398          test_epoll
    1399          test_gdbm
    1400          test_locale
    1401          test_normalization
    1402          test_ossaudiodev
    1403          test_pep277
    1404          test_tcl
    1405          test_tk
    1406          test_ttk_guionly
    1407          test_ttk_textonly
    1408          test_multiprocessing
    1409          """,
    1410      'netbsd3':
    1411          """
    1412          test_ascii_formatd
    1413          test_bsddb
    1414          test_bsddb185
    1415          test_bsddb3
    1416          test_ctypes
    1417          test_curses
    1418          test_dl
    1419          test_epoll
    1420          test_gdbm
    1421          test_locale
    1422          test_ossaudiodev
    1423          test_pep277
    1424          test_tcl
    1425          test_tk
    1426          test_ttk_guionly
    1427          test_ttk_textonly
    1428          test_multiprocessing
    1429          """,
    1430  }
    1431  _expectations['freebsd5'] = _expectations['freebsd4']
    1432  _expectations['freebsd6'] = _expectations['freebsd4']
    1433  _expectations['freebsd7'] = _expectations['freebsd4']
    1434  _expectations['freebsd8'] = _expectations['freebsd4']
    1435
    1436 -class _ExpectedSkips:
    1437 -    def __init__(self):
    1438          import os.path
    1439          from test import test_timeout
    1440
    1441          self.valid = False
    1442          if sys.platform in _expectations:
    1443              s = _expectations[sys.platform]
    1444              self.expected = set(s.split())
    1445
    1446              # expected to be skipped on every platform, even Linux
    1447              self.expected.add('test_linuxaudiodev')
    1448
    1449              if not os.path.supports_unicode_filenames:
    1450                  self.expected.add('test_pep277')
    1451
    1452              if test_timeout.skip_expected:
    1453                  self.expected.add('test_timeout')
    1454
    1455              if sys.maxint == 9223372036854775807L:
    1456                  self.expected.add('test_imageop')
    1457
    1458              if sys.platform != "darwin":
    1459                  MAC_ONLY = ["test_macos", "test_macostools", "test_aepack",
    1460                              "test_plistlib", "test_scriptpackages",
    1461                              "test_applesingle"]
    1462                  for skip in MAC_ONLY:
    1463                      self.expected.add(skip)
    1464              elif len(u'\0'.encode('unicode-internal')) == 4:
    1465                  self.expected.add("test_macostools")
    1466
    1467
    1468              if sys.platform != "win32":
    1469                  # test_sqlite is only reliable on Windows where the library
    1470                  # is distributed with Python
    1471                  WIN_ONLY = ["test_unicode_file", "test_winreg",
    1472                              "test_winsound", "test_startfile",
    1473                              "test_sqlite"]
    1474                  for skip in WIN_ONLY:
    1475                      self.expected.add(skip)
    1476
    1477              if sys.platform != 'irix':
    1478                  IRIX_ONLY = ["test_imageop", "test_al", "test_cd", "test_cl",
    1479                               "test_gl", "test_imgfile"]
    1480                  for skip in IRIX_ONLY:
    1481                      self.expected.add(skip)
    1482
    1483              if sys.platform != 'sunos5':
    1484                  self.expected.add('test_sunaudiodev')
    1485                  self.expected.add('test_nis')
    1486
    1487              if not sys.py3kwarning:
    1488                  self.expected.add('test_py3kwarn')
    1489
    1490              self.valid = True
    1491
    1492 -    def isvalid(self):
    1493          "Return true iff _ExpectedSkips knows about the current platform."
    1494          return self.valid
    1495
    1496 -    def getexpected(self):
    1497          """Return set of test names we expect to skip on current platform.
    1498
    1499          self.isvalid() must be true.
    1500          """
    1501
    1502          assert self.isvalid()
    1503          return self.expected
    1504
    1505  if __name__ == '__main__':
    1506      # findtestdir() gets the dirname out of __file__, so we have to make it
    1507      # absolute before changing the working directory.
    1508      # For example __file__ may be relative when running trace or profile.
    1509      # See issue #9323.
    1510      __file__ = os.path.abspath(__file__)
    1511
    1512      # sanity check
    1513      assert __file__ == os.path.abspath(sys.argv[0])
    1514
    1515      # When tests are run from the Python build directory, it is best practice
    1516      # to keep the test files in a subfolder.  It eases the cleanup of leftover
    1517      # files using command "make distclean".
    1518      if sysconfig.is_python_build():
    1519          TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build')
    1520          TEMPDIR = os.path.abspath(TEMPDIR)
    1521          if not os.path.exists(TEMPDIR):
    1522              os.mkdir(TEMPDIR)
    1523
    1524      # Define a writable temp dir that will be used as cwd while running
    1525      # the tests. The name of the dir includes the pid to allow parallel
    1526      # testing (see the -j option).
    1527      TESTCWD = 'test_python_{}'.format(os.getpid())
    1528
    1529      TESTCWD = os.path.join(TEMPDIR, TESTCWD)
    1530
    1531      # Run the tests in a context manager that temporary changes the CWD to a
    1532      # temporary and writable directory. If it's not possible to create or
    1533      # change the CWD, the original CWD will be used. The original CWD is
    1534      # available from test_support.SAVEDCWD.
    1535      with test_support.temp_cwd(TESTCWD, quiet=True):
    1536          main()
    1537