1
0
mirror of https://github.com/kevin1024/vcrpy.git synced 2025-12-09 01:03:24 +00:00

Compare commits

...

28 Commits

Author SHA1 Message Date
Ivan Malison
92303a911a v1.6.0 2015-07-03 10:17:44 -07:00
Ivan 'Goat' Malison
57e0e6c753 Merge pull request #164 from abhinav/tornado
Tornado support
2015-07-03 10:01:20 -07:00
Abhinav Gupta
c37d607b97 Don't install pycurl if pypy is being used.
Pycurl doesn't yet support pypy.
2015-07-02 17:14:00 -07:00
Abhinav Gupta
7922fec9fe Tornado support 2015-07-02 14:33:34 -07:00
Ivan 'Goat' Malison
7d175b0f91 Merge pull request #162 from graingert/conditional-requirements
use conditional requirements for backport libraries closes #147
2015-07-02 09:25:10 -07:00
Thomas Grainger
41949f7dc6 use conditional requirements for backport libraries Fixes #147 2015-07-02 09:34:27 +01:00
Ivan 'Goat' Malison
d14888ccd8 Merge pull request #161 from graingert/packaging-fixes
packaging fixes
2015-06-30 12:02:38 -07:00
Thomas Grainger
a9ede54064 packaging fixes 2015-06-30 12:19:05 +01:00
Ivan 'Goat' Malison
ce7ceb0a1e Merge pull request #159 from MrJohz/master
Fix issue #158
2015-06-25 01:04:12 -04:00
Jonathan
e742d32a8a Update test to ensure that filter is correctly applied 2015-06-24 16:27:45 +01:00
Jonathan
ccc1ccaa0e Allow filtering post params in requests 2015-06-24 16:23:00 +01:00
Ivan Malison
731a33a79a test for xmlrpclib to make sure #140 is actually fixed. 2015-06-09 01:13:58 -07:00
Antoine Bertin
6cbc0fb279 Fix httplib endheaders if a message_body exists
Fixes #140
2015-06-09 00:59:43 -07:00
Ivan Malison
789f118c98 whitespace change. 2015-06-09 00:54:54 -07:00
Ivan Malison
4f07cb5257 Fix tox.ini requests26: requests==2.6.0 2015-06-09 00:53:28 -07:00
Ivan Malison
ad6f635ac2 Version 1.5.2. 2015-05-15 16:54:40 -07:00
Ivan 'Goat' Malison
142b840eee Merge pull request #155 from gazpachoking/explicit_paths
Crash when cassette path contains cassette_library_dir
2015-05-15 16:51:03 -07:00
Chase Sterling
32c687522d Fix bug when specifying cassette path containing cassette_library_dir 2015-05-15 19:07:18 -04:00
Chase Sterling
5fc33c7e70 Add tests for explicitly specifying a path when library dir is defined 2015-05-15 19:06:05 -04:00
Ivan Malison
0f81f023c8 Fix readme, version 1.5.1 2015-05-14 14:46:14 -07:00
Ivan Malison
e324a9677d version 1.5.0 2015-05-14 14:05:50 -07:00
Ivan Malison
28640beb7d README updates. 2015-05-14 14:03:49 -07:00
Ivan 'Goat' Malison
c338d5d32c Merge pull request #154 from marco-santamaria/master
Filter parameters from 'application/json' content-type POST requests
2015-05-14 14:03:14 -07:00
marco.santamaria
59aa351ca8 Added support for json post data in filter_post_data_parameters. 2015-05-14 14:13:14 +02:00
Ivan Malison
2323b9da5f Automatically generate cassette names from function names. Add
`path_transformer` and `func_path_generator`. Closes #151.
2015-05-10 03:22:43 -07:00
Ivan Malison
0bbbc694b0 Make CassetteContextDecorator decorator produce reentrant functions.
Closes #150.
2015-05-09 23:14:00 -07:00
Kevin McCarthy
d293020617 Merge pull request #153 from addgene/mw/specify-six-version
Fix version of `six` dependency.
2015-05-07 20:34:38 -10:00
Morgan Wahl
daac863f0b Fixed version of six dependency.
`from six.moves.http_client import HTTPConnection` fails before version 1.5.0 of six. (on Python 2.7, at least.)
2015-05-07 15:35:26 -04:00
23 changed files with 1590 additions and 688 deletions

View File

@@ -16,6 +16,7 @@ env:
- WITH_LIB="urllib31.7"
- WITH_LIB="urllib31.9"
- WITH_LIB="urllib31.10"
- WITH_LIB="tornado"
matrix:
allow_failures:
- env: WITH_LIB="boto"
@@ -33,7 +34,7 @@ python:
- 3.4
- pypy
install:
- pip install PyYAML pytest pytest-localserver --use-mirrors
- pip install .
- if [ $WITH_LIB = "requests1.x" ] ; then pip install requests==1.2.3; fi
- if [ $WITH_LIB = "requests2.2" ] ; then pip install requests==2.2.1; fi
- if [ $WITH_LIB = "requests2.3" ] ; then pip install requests==2.3.0; fi
@@ -45,4 +46,6 @@ install:
- if [ $WITH_LIB = "urllib31.7" ] ; then pip install certifi urllib3==1.7.1; fi
- if [ $WITH_LIB = "urllib31.9" ] ; then pip install certifi urllib3==1.9.1; fi
- if [ $WITH_LIB = "urllib31.10" ] ; then pip install certifi urllib3==1.10.2; fi
- if [ $WITH_LIB = "tornado" ] ; then pip install tornado==4.2 pytest-tornado; fi
- if [ $WITH_LIB = "tornado" -a $TRAVIS_PYTHON_VERSION != "pypy" ] ; then pip install pycurl; fi
script: python setup.py test

View File

@@ -1,4 +1,4 @@
include README.md
include README.rst
include LICENSE.txt
include tox.ini
recursive-include tests *

596
README.md
View File

@@ -1,596 +0,0 @@
# VCR.py
![vcr.py](https://raw.github.com/kevin1024/vcrpy/master/vcr.png)
This is a Python version of [Ruby's VCR library](https://github.com/vcr/vcr).
[![Build Status](https://secure.travis-ci.org/kevin1024/vcrpy.png?branch=master)](http://travis-ci.org/kevin1024/vcrpy)
[![Stories in Ready](https://badge.waffle.io/kevin1024/vcrpy.png?label=ready&title=Ready)](https://waffle.io/kevin1024/vcrpy)
## What it does
Simplify and speed up testing HTTP by recording all HTTP interactions and
saving them to "cassette" files, which are yaml files containing the contents
of your requests and responses. Then when you run your tests again, they all
just hit the text files instead of the internet. This speeds up your tests and
lets you work offline.
If the server you are testing against ever changes its API, all you need to do
is delete your existing cassette files, and run your tests again. All of the
mocked responses will be updated with the new API.
## Compatibility Notes
VCR.py supports Python 2.6 and 2.7, 3.3, 3.4, and [pypy](http://pypy.org).
The following http libraries are supported:
* urllib2
* urllib3
* http.client (python3)
* requests (both 1.x and 2.x versions)
* httplib2
* boto
## Usage
```python
import vcr
import urllib2
with vcr.use_cassette('fixtures/vcr_cassettes/synopsis.yaml'):
response = urllib2.urlopen('http://www.iana.org/domains/reserved').read()
assert 'Example domains' in response
```
Run this test once, and VCR.py will record the HTTP request to
`fixtures/vcr_cassettes/synopsis.yml`. Run it again, and VCR.py will replay the
response from iana.org when the http request is made. This test is now fast (no
real HTTP requests are made anymore), deterministic (the test will continue to
pass, even if you are offline, or iana.org goes down for maintenance) and
accurate (the response will contain the same headers and body you get from a
real request).
You can also use VCR.py as a decorator. The same request above would look like
this:
```python
@vcr.use_cassette('fixtures/vcr_cassettes/synopsis.yaml')
def test_iana():
response = urllib2.urlopen('http://www.iana.org/domains/reserved').read()
assert 'Example domains' in response
```
All of the parameters and configuration works the same for the decorator
version.
## Configuration
If you don't like VCR's defaults, you can set options by instantiating a `VCR`
class and setting the options on it.
```python
import vcr
my_vcr = vcr.VCR(
serializer = 'json',
cassette_library_dir = 'fixtures/cassettes',
record_mode = 'once',
match_on = ['uri', 'method'],
)
with my_vcr.use_cassette('test.json'):
# your http code here
```
Otherwise, you can override options each time you use a cassette.
```python
with vcr.use_cassette('test.yml', serializer='json', record_mode='once'):
# your http code here
```
Note: Per-cassette overrides take precedence over the global config.
## Request matching
Request matching is configurable and allows you to change which requests VCR
considers identical. The default behavior is
`['method', 'scheme', 'host', 'port', 'path', 'query']` which means that
requests with both the same URL and method (ie POST or GET) are considered
identical.
This can be configured by changing the `match_on` setting.
The following options are available :
* method (for example, POST or GET)
* uri (the full URI.)
* host (the hostname of the server receiving the request)
* port (the port of the server receiving the request)
* path (the path of the request)
* query (the query string of the request)
* body (the entire request body)
* headers (the headers of the request)
Backwards compatible matchers:
* url (the `uri` alias)
If these options don't work for you, you can also register your own request
matcher. This is described in the Advanced section of this README.
## Record Modes
VCR supports 4 record modes (with the same behavior as Ruby's VCR):
### once
* Replay previously recorded interactions.
* Record new interactions if there is no cassette file.
* Cause an error to be raised for new requests if there is a cassette file.
It is similar to the new_episodes record mode, but will prevent new, unexpected
requests from being made (i.e. because the request URI changed).
once is the default record mode, used when you do not set one.
### new_episodes
* Record new interactions.
* Replay previously recorded interactions. It is similar to the once record
mode, but will always record new interactions, even if you have an existing
recorded one that is similar, but not identical.
This was the default behavior in versions < 0.3.0
### none
* Replay previously recorded interactions.
* Cause an error to be raised for any new requests. This is useful when your
code makes potentially dangerous HTTP requests. The none record mode
guarantees that no new HTTP requests will be made.
### all
* Record new interactions.
* Never replay previously recorded interactions. This can be temporarily used
to force VCR to re-record a cassette (i.e. to ensure the responses are not
out of date) or can be used when you simply want to log all HTTP requests.
## Advanced Features
If you want, VCR.py can return information about the cassette it is using to
record your requests and responses. This will let you record your requests and
responses and make assertions on them, to make sure that your code under test
is generating the expected requests and responses. This feature is not present
in Ruby's VCR, but I think it is a nice addition. Here's an example:
```python
import vcr
import urllib2
with vcr.use_cassette('fixtures/vcr_cassettes/synopsis.yaml') as cass:
response = urllib2.urlopen('http://www.zombo.com/').read()
# cass should have 1 request inside it
assert len(cass) == 1
# the request uri should have been http://www.zombo.com/
assert cass.requests[0].uri == 'http://www.zombo.com/'
```
The `Cassette` object exposes the following properties which I consider part of
the API. The fields are as follows:
* `requests`: A list of vcr.Request objects corresponding to the http requests
that were made during the recording of the cassette. The requests appear in the
order that they were originally processed.
* `responses`: A list of the responses made.
* `play_count`: The number of times this cassette has played back a response.
* `all_played`: A boolean indicating whether all the responses have been
played back.
* `responses_of(request)`: Access the responses that match a given request
The `Request` object has the following properties:
* `uri`: The full uri of the request. Example: "https://google.com/?q=vcrpy"
* `scheme`: The scheme used to make the request (http or https)
* `host`: The host of the request, for example "www.google.com"
* `port`: The port the request was made on
* `path`: The path of the request. For example "/" or "/home.html"
* `query`: The parsed query string of the request. Sorted list of name, value pairs.
* `method` : The method used to make the request, for example "GET" or "POST"
* `body`: The body of the request, usually empty except for POST / PUT / etc
Backwards compatible properties:
* `url`: The `uri` alias
* `protocol`: The `scheme` alias
## Register your own serializer
Don't like JSON or YAML? That's OK, VCR.py can serialize to any format you
would like. Create your own module or class instance with 2 methods:
* `def deserialize(cassette_string)`
* `def serialize(cassette_dict)`
Finally, register your class with VCR to use your new serializer.
```python
import vcr
class BogoSerializer(object):
"""
Must implement serialize() and deserialize() methods
"""
pass
my_vcr = vcr.VCR()
my_vcr.register_serializer('bogo', BogoSerializer())
with my_vcr.use_cassette('test.bogo', serializer='bogo'):
# your http here
# After you register, you can set the default serializer to your new serializer
my_vcr.serializer = 'bogo'
with my_vcr.use_cassette('test.bogo'):
# your http here
```
## Register your own request matcher
Create your own method with the following signature
```python
def my_matcher(r1, r2):
```
Your method receives the two requests and must return `True` if they match,
`False` if they don't.
Finally, register your method with VCR to use your new request matcher.
```python
import vcr
def jurassic_matcher(r1, r2):
return r1.uri == r2.uri and 'JURASSIC PARK' in r1.body
my_vcr = vcr.VCR()
my_vcr.register_matcher('jurassic', jurassic_matcher)
with my_vcr.use_cassette('test.yml', match_on=['jurassic']):
# your http here
# After you register, you can set the default match_on to use your new matcher
my_vcr.match_on = ['jurassic']
with my_vcr.use_cassette('test.yml'):
# your http here
```
## Filter sensitive data from the request
If you are checking your cassettes into source control, and are using some form
of authentication in your tests, you can filter out that information so it won't
appear in your cassette files. There are a few ways to do this:
### Filter information from HTTP Headers
Use the `filter_headers` configuration option with a list of headers to filter.
```python
with my_vcr.use_cassette('test.yml', filter_headers=['authorization']):
# sensitive HTTP request goes here
```
### Filter information from HTTP querystring
Use the `filter_query_parameters` configuration option with a list of query
parameters to filter.
```python
with my_vcr.use_cassette('test.yml', filter_query_parameters=['api_key']):
requests.get('http://api.com/getdata?api_key=secretstring')
```
### Filter information from HTTP post data
Use the `filter_post_data_parameters` configuration option with a list of post data
parameters to filter.
```python
with my_vcr.use_cassette('test.yml', filter_post_data_parameters=['client_secret']):
requests.post('http://api.com/postdata', data={'api_key': 'secretstring'})
```
### Custom Request filtering
If none of these covers your request filtering needs, you can register a callback
that will manipulate the HTTP request before adding it to the cassette. Use the
`before_record` configuration option to so this. Here is an example that will
never record requests to the /login endpoint.
```python
def before_record_cb(request):
if request.path != '/login':
return request
my_vcr = vcr.VCR(
before_record = before_record_cb,
)
with my_vcr.use_cassette('test.yml'):
# your http code here
```
You can also mutate the response using this callback. For example, you could
remove all query parameters from any requests to the `'/login'` path.
```python
def scrub_login_request(request):
if request.path == '/login':
request.uri, _ = urllib.splitquery(response.uri)
return request
my_vcr = vcr.VCR(
before_record=scrub_login_request,
)
with my_vcr.use_cassette('test.yml'):
# your http code here
```
### Custom Response Filtering
VCR.py also suports response filtering with the `before_record_response` keyword
argument. It's usage is similar to that of `before_record`:
```python
def scrub_string(string, replacement=''):
def before_record_reponse(response):
return response['body']['string'] = response['body']['string'].replace(string, replacement)
return scrub_string
my_vcr = vcr.VCR(
before_record=scrub_string(settings.USERNAME, 'username'),
)
with my_vcr.use_cassette('test.yml'):
# your http code here
```
## Ignore requests
If you would like to completely ignore certain requests, you can do it in a
few ways:
* Set the `ignore_localhost` option equal to True. This will not record any
requests sent to (or responses from) localhost, 127.0.0.1, or 0.0.0.0.
* Set the `ignore_hosts` configuration option to a list of hosts to ignore
* Add a `before_record` callback that returns None for requests you want to
ignore
Requests that are ignored by VCR will not be saved in a cassette, nor played
back from a cassette. VCR will completely ignore those requests as if it
didn't notice them at all, and they will continue to hit the server as if VCR
were not there.
## Custom Patches
If you use a custom `HTTPConnection` class, or otherwise make http
requests in a way that requires additional patching, you can use the
`custom_patches` keyword argument of the `VCR` and `Cassette` objects
to patch those objects whenever a cassette's context is entered. To
patch a custom version of `HTTPConnection` you can do something like
this:
```
import where_the_custom_https_connection_lives
from vcr.stubs import VCRHTTPSConnection
my_vcr = config.VCR(custom_patches=((where_the_custom_https_connection_lives, 'CustomHTTPSConnection', VCRHTTPSConnection),))
@my_vcr.use_cassette(...)
```
## Installation
VCR.py is a package on PyPI, so you can `pip install vcrpy` (first you may need
to `brew install libyaml` [[Homebrew](http://mxcl.github.com/homebrew/)])
## Ruby VCR compatibility
VCR.py does not aim to match the format of the Ruby VCR YAML files. Cassettes
generated by Ruby's VCR are not compatible with VCR.py.
## Running VCR's test suite
The tests are all run automatically on [Travis
CI](https://travis-ci.org/kevin1024/vcrpy), but you can also run them yourself
using [py.test](http://pytest.org/) and [Tox](http://tox.testrun.org/). Tox
will automatically run them in all environments VCR.py supports. The test
suite is pretty big and slow, but you can tell tox to only run specific tests
like this:
`tox -e py27requests -- -v -k "'test_status_code or test_gzip'"`
This will run only tests that look like `test_status_code` or `test_gzip` in
the test suite, and only in the python 2.7 environment that has `requests`
installed.
Also, in order for the boto tests to run, you will need an AWS key. Refer to
the [boto
documentation](http://boto.readthedocs.org/en/latest/getting_started.html) for
how to set this up. I have marked the boto tests as optional in Travis so you
don't have to worry about them failing if you submit a pull request.
## Logging
VCR.py has a few log messages you can turn on to help you figure out if HTTP
requests are hitting a real server or not. You can turn them on like this:
```python
import vcr
import requests
import logging
logging.basicConfig() # you need to initialize logging, otherwise you will not see anything from vcrpy
vcr_log = logging.getLogger("vcr")
vcr_log.setLevel(logging.INFO)
with vcr.use_cassette('headers.yml'):
requests.get('http://httpbin.org/headers')
```
The first time you run this, you will see:
```
INFO:vcr.stubs:<Request (GET) http://httpbin.org/headers> not in cassette, sending to real server
```
The second time, you will see:
```
INFO:vcr.stubs:Playing response for <Request (GET) http://httpbin.org/headers> from cassette
```
If you set the loglevel to DEBUG, you will also get information about which
matchers didn't match. This can help you with debugging custom matchers.
## Upgrade
### New Cassette Format
The cassette format has changed in _VCR.py 1.x_, the _VCR.py 0.x_ cassettes
cannot be used with _VCR.py 1.x_. The easiest way to upgrade is to simply
delete your cassettes and re-record all of them. VCR.py also provides a
migration script that attempts to upgrade your 0.x cassettes to the new 1.x
format. To use it, run the following command:
```
python -m vcr.migration PATH
```
The PATH can be either a path to the directory with cassettes or the path to a
single cassette.
*Note*: Back up your cassettes files before migration.
The migration *should* only modify cassettes using the old 0.x format.
## New serializer / deserializer API
If you made a custom serializer, you will need to update it to match the new
API in version 1.0.x
* Serializers now take dicts and return strings.
* Deserializers take strings and return dicts (instead of requests, responses
pair)
## Changelog
* 1.4.2 Fix a bug caused by requests 2.7 and chunked transfer encoding
* 1.4.1 Include README, tests, LICENSE in package. Thanks @ralphbean.
* 1.4.0 Filter post data parameters (thanks @eadmundo), support for
posting files through requests, inject_cassette kwarg to access
cassette from `use_cassette` decorated function,
`with_current_defaults` actually works (thanks @samstav).
* 1.3.0 Fix/add support for urllib3 (thanks @aisch), fix default
port for https (thanks @abhinav).
* 1.2.0 Add custom_patches argument to VCR/Cassette objects to allow
users to stub custom classes when cassettes become active.
* 1.1.4 Add force reset around calls to actual connection from stubs, to ensure
compatibility with the version of httplib/urlib2 in python 2.7.9.
* 1.1.3 Fix python3 headers field (thanks @rtaboada), fix boto test (thanks
@telaviv), fix new_episodes record mode (thanks @jashugan), fix Windows
connectionpool stub bug (thanks @gazpachoking), add support for requests 2.5
* 1.1.2 Add urllib==1.7.1 support. Make json serialize error handling correct
Improve logging of match failures.
* 1.1.1 Use function signature preserving `wrapt.decorator` to write the
decorator version of use_cassette in order to ensure compatibility with
py.test fixtures and python 2. Move all request filtering into the
`before_record_callable`.
* 1.1.0 Add `before_record_response`. Fix several bugs related to the context
management of cassettes.
* 1.0.3: Fix an issue with requests 2.4 and make sure case sensitivity is
consistent across python versions
* 1.0.2: Fix an issue with requests 2.3
* 1.0.1: Fix a bug with the new ignore requests feature and the once
record mode
* 1.0.0: _BACKWARDS INCOMPATIBLE_: Please see the 'upgrade' section in the
README. Take a look at the matcher section as well, you might want to
update your `match_on` settings. Add support for filtering sensitive
data from requests, matching query strings after the order changes and
improving the built-in matchers, (thanks to @mshytikov), support for
ignoring requests to certain hosts, bump supported Python3 version to
3.4, fix some bugs with Boto support (thanks @marusich), fix error with
URL field capitalization in README (thanks @simon-weber), added some log
messages to help with debugging, added `all_played` property on cassette
(thanks @mshytikov)
* 0.7.0: VCR.py now supports Python 3! (thanks @asundg) Also I refactored
the stub connections quite a bit to add support for the putrequest and
putheader calls. This version also adds support for httplib2 (thanks
@nilp0inter). I have added a couple tests for boto since it is an http
client in its own right. Finally, this version includes a fix for a bug
where requests wasn't being patched properly (thanks @msabramo).
* 0.6.0: Store response headers as a list since a HTTP response can have the
same header twice (happens with set-cookie sometimes). This has the added
benefit of preserving the order of headers. Thanks @smallcode for the bug
report leading to this change. I have made an effort to ensure backwards
compatibility with the old cassettes' header storage mechanism, but if you
want to upgrade to the new header storage, you should delete your
cassettes and re-record them. Also this release adds better error messages
(thanks @msabramo) and adds support for using VCR as a decorator (thanks
@smallcode for the motivation)
* 0.5.0: Change the `response_of` method to `responses_of` since cassettes
can now contain more than one response for a request. Since this changes
the API, I'm bumping the version. Also includes 2 bugfixes: a better error
message when attempting to overwrite a cassette file, and a fix for a bug
with requests sessions (thanks @msabramo)
* 0.4.0: Change default request recording behavior for multiple requests. If
you make the same request multiple times to the same URL, the response
might be different each time (maybe the response has a timestamp in it or
something), so this will make the same request multiple times and save them
all. Then, when you are replaying the cassette, the responses will be
played back in the same order in which they were received. If you were
making multiple requests to the same URL in a cassette before version
0.4.0, you might need to regenerate your cassette files. Also, removes
support for the cassette.play_count counter API, since individual requests
aren't unique anymore. A cassette might contain the same request several
times. Also removes secure overwrite feature since that was breaking
overwriting files in Windows, and fixes a bug preventing request's
automatic body decompression from working.
* 0.3.5: Fix compatibility with requests 2.x
* 0.3.4: Bugfix: close file before renaming it. This fixes an issue on
Windows. Thanks @smallcode for the fix.
* 0.3.3: Bugfix for error message when an unreigstered custom matcher was
used
* 0.3.2: Fix issue with new config syntax and the `match_on` parameter.
Thanks, @chromy!
* 0.3.1: Fix issue causing full paths to be sent on the HTTP request line.
* 0.3.0: *Backwards incompatible release* - Added support for record modes,
and changed the default recording behavior to the "once" record mode.
Please see the documentation on record modes for more. Added support for
custom request matching, and changed the default request matching behavior to
match only on the URL and method. Also, improved the httplib mocking to add
support for the `HTTPConnection.send()` method. This means that requests won't
actually be sent until the response is read, since I need to record the entire
request in order to match up the appropriate response. I don't think this
should cause any issues unless you are sending requests without ever loading
the response (which none of the standard httplib wrappers do, as far as I know.
Thanks to @fatuhoku for some of the ideas and the motivation behind this
release.
* 0.2.1: Fixed missing modules in setup.py
* 0.2.0: Added configuration API, which lets you configure some settings on
VCR (see the README). Also, VCR no longer saves cassettes if they haven't
changed at all and supports JSON as well as YAML (thanks @sirpengi). Added
amazing new skeumorphic logo, thanks @hairarrow.
* 0.1.0: *backwards incompatible release - delete your old cassette files*:
This release adds the ability to access the cassette to make assertions on
it, as well as a major code refactor thanks to @dlecocq. It also fixes a
couple longstanding bugs with redirects and HTTPS. [#3 and #4]
* 0.0.4: If you have libyaml installed, vcrpy will use the c bindings
instead. Speed up your tests! Thanks @dlecocq
* 0.0.3: Add support for requests 1.2.3. Support for older versions of
requests dropped (thanks @vitormazzi and @bryanhelmig)
* 0.0.2: Add support for requests / urllib3
* 0.0.1: Initial Release
# License
This library uses the MIT license. See [LICENSE.txt](LICENSE.txt) for more details

739
README.rst Normal file
View File

@@ -0,0 +1,739 @@
VCR.py
======
.. figure:: https://raw.github.com/kevin1024/vcrpy/master/vcr.png
:alt: vcr.py
vcr.py
This is a Python version of `Ruby's VCR
library <https://github.com/vcr/vcr>`__.
|Build Status| |Stories in Ready|
What it does
------------
VCR.py simplifies and speeds up tests that make HTTP requests. The first
time you run code that is inside a VCR.py context manager or decorated
function, VCR.py records all HTTP interactions that take place through
the libraries it supports and serializes and writes them to a flat file
(in yaml format by default). This flat file is called a cassette. When
the relevant peice of code is executed again, VCR.py will read the
serialized requests and responses from the aforementioned cassette file,
and intercept any HTTP requests that it recognizes from the original
test run and return responses that corresponded to those requests. This
means that the requests will not actually result in HTTP traffic, which
confers several benefits including:
- The ability to work offline
- Completely deterministic tests
- Increased test execution speed
If the server you are testing against ever changes its API, all you need
to do is delete your existing cassette files, and run your tests again.
VCR.py will detect the absence of a cassette file and once again record
all HTTP interactions, which will update them to correspond to the new
API.
Compatibility Notes
-------------------
VCR.py supports Python 2.6 and 2.7, 3.3, 3.4, and
`pypy <http://pypy.org>`__.
The following http libraries are supported:
- urllib2
- urllib3
- http.client (python3)
- requests (both 1.x and 2.x versions)
- httplib2
- boto
Usage
-----
.. code:: python
import vcr
import urllib2
with vcr.use_cassette('fixtures/vcr_cassettes/synopsis.yaml'):
response = urllib2.urlopen('http://www.iana.org/domains/reserved').read()
assert 'Example domains' in response
Run this test once, and VCR.py will record the HTTP request to
``fixtures/vcr_cassettes/synopsis.yml``. Run it again, and VCR.py will
replay the response from iana.org when the http request is made. This
test is now fast (no real HTTP requests are made anymore), deterministic
(the test will continue to pass, even if you are offline, or iana.org
goes down for maintenance) and accurate (the response will contain the
same headers and body you get from a real request).
You can also use VCR.py as a decorator. The same request above would
look like this:
.. code:: python
@vcr.use_cassette('fixtures/vcr_cassettes/synopsis.yaml')
def test_iana():
response = urllib2.urlopen('http://www.iana.org/domains/reserved').read()
assert 'Example domains' in response
When using the decorator version of ``use_cassette``, it is possible to
omit the path to the cassette file.
.. code:: python
@vcr.use_cassette()
def test_iana():
response = urllib2.urlopen('http://www.iana.org/domains/reserved').read()
assert 'Example domains' in response
In this case, the cassette file will be given the same name as the test
function, and it will be placed in the same directory as the file in
which the test is defined. See the Automatic Test Naming section below
for more details.
Configuration
-------------
If you don't like VCR's defaults, you can set options by instantiating a
``VCR`` class and setting the options on it.
.. code:: python
import vcr
my_vcr = vcr.VCR(
serializer = 'json',
cassette_library_dir = 'fixtures/cassettes',
record_mode = 'once',
match_on = ['uri', 'method'],
)
with my_vcr.use_cassette('test.json'):
# your http code here
Otherwise, you can override options each time you use a cassette.
.. code:: python
with vcr.use_cassette('test.yml', serializer='json', record_mode='once'):
# your http code here
Note: Per-cassette overrides take precedence over the global config.
Request matching
----------------
Request matching is configurable and allows you to change which requests
VCR considers identical. The default behavior is
``['method', 'scheme', 'host', 'port', 'path', 'query']`` which means
that requests with both the same URL and method (ie POST or GET) are
considered identical.
This can be configured by changing the ``match_on`` setting.
The following options are available :
- method (for example, POST or GET)
- uri (the full URI.)
- host (the hostname of the server receiving the request)
- port (the port of the server receiving the request)
- path (the path of the request)
- query (the query string of the request)
- body (the entire request body)
- headers (the headers of the request)
Backwards compatible matchers:
- url (the ``uri`` alias)
If these options don't work for you, you can also register your own
request matcher. This is described in the Advanced section of this
README.
Record Modes
------------
VCR supports 4 record modes (with the same behavior as Ruby's VCR):
once
~~~~
- Replay previously recorded interactions.
- Record new interactions if there is no cassette file.
- Cause an error to be raised for new requests if there is a cassette
file.
It is similar to the new\_episodes record mode, but will prevent new,
unexpected requests from being made (i.e. because the request URI
changed).
once is the default record mode, used when you do not set one.
new\_episodes
~~~~~~~~~~~~~
- Record new interactions.
- Replay previously recorded interactions. It is similar to the once
record mode, but will always record new interactions, even if you
have an existing recorded one that is similar, but not identical.
This was the default behavior in versions < 0.3.0
none
~~~~
- Replay previously recorded interactions.
- Cause an error to be raised for any new requests. This is useful when
your code makes potentially dangerous HTTP requests. The none record
mode guarantees that no new HTTP requests will be made.
all
~~~
- Record new interactions.
- Never replay previously recorded interactions. This can be
temporarily used to force VCR to re-record a cassette (i.e. to ensure
the responses are not out of date) or can be used when you simply
want to log all HTTP requests.
Advanced Features
-----------------
If you want, VCR.py can return information about the cassette it is
using to record your requests and responses. This will let you record
your requests and responses and make assertions on them, to make sure
that your code under test is generating the expected requests and
responses. This feature is not present in Ruby's VCR, but I think it is
a nice addition. Here's an example:
.. code:: python
import vcr
import urllib2
with vcr.use_cassette('fixtures/vcr_cassettes/synopsis.yaml') as cass:
response = urllib2.urlopen('http://www.zombo.com/').read()
# cass should have 1 request inside it
assert len(cass) == 1
# the request uri should have been http://www.zombo.com/
assert cass.requests[0].uri == 'http://www.zombo.com/'
The ``Cassette`` object exposes the following properties which I
consider part of the API. The fields are as follows:
- ``requests``: A list of vcr.Request objects corresponding to the http
requests that were made during the recording of the cassette. The
requests appear in the order that they were originally processed.
- ``responses``: A list of the responses made.
- ``play_count``: The number of times this cassette has played back a
response.
- ``all_played``: A boolean indicating whether all the responses have
been played back.
- ``responses_of(request)``: Access the responses that match a given
request
The ``Request`` object has the following properties:
- ``uri``: The full uri of the request. Example:
"https://google.com/?q=vcrpy"
- ``scheme``: The scheme used to make the request (http or https)
- ``host``: The host of the request, for example "www.google.com"
- ``port``: The port the request was made on
- ``path``: The path of the request. For example "/" or "/home.html"
- ``query``: The parsed query string of the request. Sorted list of
name, value pairs.
- ``method`` : The method used to make the request, for example "GET"
or "POST"
- ``body``: The body of the request, usually empty except for POST /
PUT / etc
Backwards compatible properties:
- ``url``: The ``uri`` alias
- ``protocol``: The ``scheme`` alias
Register your own serializer
----------------------------
Don't like JSON or YAML? That's OK, VCR.py can serialize to any format
you would like. Create your own module or class instance with 2 methods:
- ``def deserialize(cassette_string)``
- ``def serialize(cassette_dict)``
Finally, register your class with VCR to use your new serializer.
.. code:: python
import vcr
class BogoSerializer(object):
"""
Must implement serialize() and deserialize() methods
"""
pass
my_vcr = vcr.VCR()
my_vcr.register_serializer('bogo', BogoSerializer())
with my_vcr.use_cassette('test.bogo', serializer='bogo'):
# your http here
# After you register, you can set the default serializer to your new serializer
my_vcr.serializer = 'bogo'
with my_vcr.use_cassette('test.bogo'):
# your http here
Register your own request matcher
---------------------------------
Create your own method with the following signature
.. code:: python
def my_matcher(r1, r2):
Your method receives the two requests and must return ``True`` if they
match, ``False`` if they don't.
Finally, register your method with VCR to use your new request matcher.
.. code:: python
import vcr
def jurassic_matcher(r1, r2):
return r1.uri == r2.uri and 'JURASSIC PARK' in r1.body
my_vcr = vcr.VCR()
my_vcr.register_matcher('jurassic', jurassic_matcher)
with my_vcr.use_cassette('test.yml', match_on=['jurassic']):
# your http here
# After you register, you can set the default match_on to use your new matcher
my_vcr.match_on = ['jurassic']
with my_vcr.use_cassette('test.yml'):
# your http here
Filter sensitive data from the request
--------------------------------------
If you are checking your cassettes into source control, and are using
some form of authentication in your tests, you can filter out that
information so it won't appear in your cassette files. There are a few
ways to do this:
Filter information from HTTP Headers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the ``filter_headers`` configuration option with a list of headers
to filter.
.. code:: python
with my_vcr.use_cassette('test.yml', filter_headers=['authorization']):
# sensitive HTTP request goes here
Filter information from HTTP querystring
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the ``filter_query_parameters`` configuration option with a list of
query parameters to filter.
.. code:: python
with my_vcr.use_cassette('test.yml', filter_query_parameters=['api_key']):
requests.get('http://api.com/getdata?api_key=secretstring')
Filter information from HTTP post data
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the ``filter_post_data_parameters`` configuration option with a list
of post data parameters to filter.
.. code:: python
with my_vcr.use_cassette('test.yml', filter_post_data_parameters=['client_secret']):
requests.post('http://api.com/postdata', data={'api_key': 'secretstring'})
Custom Request filtering
~~~~~~~~~~~~~~~~~~~~~~~~
If none of these covers your request filtering needs, you can register a
callback that will manipulate the HTTP request before adding it to the
cassette. Use the ``before_record`` configuration option to so this.
Here is an example that will never record requests to the /login
endpoint.
.. code:: python
def before_record_cb(request):
if request.path != '/login':
return request
my_vcr = vcr.VCR(
before_record = before_record_cb,
)
with my_vcr.use_cassette('test.yml'):
# your http code here
You can also mutate the response using this callback. For example, you
could remove all query parameters from any requests to the ``'/login'``
path.
.. code:: python
def scrub_login_request(request):
if request.path == '/login':
request.uri, _ = urllib.splitquery(response.uri)
return request
my_vcr = vcr.VCR(
before_record=scrub_login_request,
)
with my_vcr.use_cassette('test.yml'):
# your http code here
Custom Response Filtering
~~~~~~~~~~~~~~~~~~~~~~~~~
VCR.py also suports response filtering with the
``before_record_response`` keyword argument. It's usage is similar to
that of ``before_record``:
.. code:: python
def scrub_string(string, replacement=''):
def before_record_reponse(response):
return response['body']['string'] = response['body']['string'].replace(string, replacement)
return scrub_string
my_vcr = vcr.VCR(
before_record=scrub_string(settings.USERNAME, 'username'),
)
with my_vcr.use_cassette('test.yml'):
# your http code here
Ignore requests
---------------
If you would like to completely ignore certain requests, you can do it
in a few ways:
- Set the ``ignore_localhost`` option equal to True. This will not
record any requests sent to (or responses from) localhost, 127.0.0.1,
or 0.0.0.0.
- Set the ``ignore_hosts`` configuration option to a list of hosts to
ignore
- Add a ``before_record`` callback that returns None for requests you
want to ignore
Requests that are ignored by VCR will not be saved in a cassette, nor
played back from a cassette. VCR will completely ignore those requests
as if it didn't notice them at all, and they will continue to hit the
server as if VCR were not there.
Custom Patches
--------------
If you use a custom ``HTTPConnection`` class, or otherwise make http
requests in a way that requires additional patching, you can use the
``custom_patches`` keyword argument of the ``VCR`` and ``Cassette``
objects to patch those objects whenever a cassette's context is entered.
To patch a custom version of ``HTTPConnection`` you can do something
like this:
::
import where_the_custom_https_connection_lives
from vcr.stubs import VCRHTTPSConnection
my_vcr = config.VCR(custom_patches=((where_the_custom_https_connection_lives, 'CustomHTTPSConnection', VCRHTTPSConnection),))
@my_vcr.use_cassette(...)
Automatic Cassette Naming
-------------------------
VCR.py now allows the omission of the path argument to the use\_cassette
function. Both of the following are now legal/should work
.. code:: python
@my_vcr.use_cassette
def my_test_function():
...
.. code:: python
@my_vcr.use_cassette()
def my_test_function():
...
In both cases, VCR.py will use a path that is generated from the
provided test function's name. If no ``cassette_library_dir`` has been
set, the cassette will be in a file with the name of the test function
in directory of the file in which the test function is declared. If a
``cassette_library_dir`` has been set, the cassette will appear in that
directory in a file with the name of the decorated function.
It is possible to control the path produced by the automatic naming
machinery by customizing the ``path_transformer`` and
``func_path_generator`` vcr variables. To add an extension to all
cassette names, use ``VCR.ensure_suffix`` as follows:
.. code:: python
my_vcr = VCR(path_transformer=VCR.ensure_suffix('.yaml'))
@my_vcr.use_cassette
def my_test_function():
Installation
------------
VCR.py is a package on PyPI, so you can ``pip install vcrpy`` (first you
may need to ``brew install libyaml``
[`Homebrew <http://mxcl.github.com/homebrew/>`__\ ])
Ruby VCR compatibility
----------------------
VCR.py does not aim to match the format of the Ruby VCR YAML files.
Cassettes generated by Ruby's VCR are not compatible with VCR.py.
Running VCR's test suite
------------------------
The tests are all run automatically on `Travis
CI <https://travis-ci.org/kevin1024/vcrpy>`__, but you can also run them
yourself using `py.test <http://pytest.org/>`__ and
`Tox <http://tox.testrun.org/>`__. Tox will automatically run them in
all environments VCR.py supports. The test suite is pretty big and slow,
but you can tell tox to only run specific tests like this:
``tox -e py27requests -- -v -k "'test_status_code or test_gzip'"``
This will run only tests that look like ``test_status_code`` or
``test_gzip`` in the test suite, and only in the python 2.7 environment
that has ``requests`` installed.
Also, in order for the boto tests to run, you will need an AWS key.
Refer to the `boto
documentation <http://boto.readthedocs.org/en/latest/getting_started.html>`__
for how to set this up. I have marked the boto tests as optional in
Travis so you don't have to worry about them failing if you submit a
pull request.
Logging
-------
VCR.py has a few log messages you can turn on to help you figure out if
HTTP requests are hitting a real server or not. You can turn them on
like this:
.. code:: python
import vcr
import requests
import logging
logging.basicConfig() # you need to initialize logging, otherwise you will not see anything from vcrpy
vcr_log = logging.getLogger("vcr")
vcr_log.setLevel(logging.INFO)
with vcr.use_cassette('headers.yml'):
requests.get('http://httpbin.org/headers')
The first time you run this, you will see:
::
INFO:vcr.stubs:<Request (GET) http://httpbin.org/headers> not in cassette, sending to real server
The second time, you will see:
::
INFO:vcr.stubs:Playing response for <Request (GET) http://httpbin.org/headers> from cassette
If you set the loglevel to DEBUG, you will also get information about
which matchers didn't match. This can help you with debugging custom
matchers.
Upgrade
-------
New Cassette Format
~~~~~~~~~~~~~~~~~~~
The cassette format has changed in *VCR.py 1.x*, the *VCR.py 0.x*
cassettes cannot be used with *VCR.py 1.x*. The easiest way to upgrade
is to simply delete your cassettes and re-record all of them. VCR.py
also provides a migration script that attempts to upgrade your 0.x
cassettes to the new 1.x format. To use it, run the following command:
::
python -m vcr.migration PATH
The PATH can be either a path to the directory with cassettes or the
path to a single cassette.
*Note*: Back up your cassettes files before migration. The migration
*should* only modify cassettes using the old 0.x format.
New serializer / deserializer API
---------------------------------
If you made a custom serializer, you will need to update it to match the
new API in version 1.0.x
- Serializers now take dicts and return strings.
- Deserializers take strings and return dicts (instead of requests,
responses pair)
Changelog
---------
- 1.6.0 [#120] Tornado support thanks (thanks @abhinav), [#147] packaging fixes
(thanks @graingert), [#158] allow filtering post params in requests
(thanks @MrJohz), [#140] add xmlrpclib support (thanks @Diaoul).
- 1.5.2 Fix crash when cassette path contains cassette library
directory (thanks @gazpachoking).
- 1.5.0 Automatic cassette naming and 'application/json' post data
filtering (thanks @marco-santamaria).
- 1.4.2 Fix a bug caused by requests 2.7 and chunked transfer encoding
- 1.4.1 Include README, tests, LICENSE in package. Thanks @ralphbean.
- 1.4.0 Filter post data parameters (thanks @eadmundo), support for
posting files through requests, inject\_cassette kwarg to access
cassette from ``use_cassette`` decorated function,
``with_current_defaults`` actually works (thanks @samstav).
- 1.3.0 Fix/add support for urllib3 (thanks @aisch), fix default port
for https (thanks @abhinav).
- 1.2.0 Add custom\_patches argument to VCR/Cassette objects to allow
users to stub custom classes when cassettes become active.
- 1.1.4 Add force reset around calls to actual connection from stubs,
to ensure compatibility with the version of httplib/urlib2 in python
2.7.9.
- 1.1.3 Fix python3 headers field (thanks @rtaboada), fix boto test
(thanks @telaviv), fix new\_episodes record mode (thanks @jashugan),
fix Windows connectionpool stub bug (thanks @gazpachoking), add
support for requests 2.5
- 1.1.2 Add urllib==1.7.1 support. Make json serialize error handling
correct Improve logging of match failures.
- 1.1.1 Use function signature preserving ``wrapt.decorator`` to write
the decorator version of use\_cassette in order to ensure
compatibility with py.test fixtures and python 2. Move all request
filtering into the ``before_record_callable``.
- 1.1.0 Add ``before_record_response``. Fix several bugs related to the
context management of cassettes.
- 1.0.3: Fix an issue with requests 2.4 and make sure case sensitivity
is consistent across python versions
- 1.0.2: Fix an issue with requests 2.3
- 1.0.1: Fix a bug with the new ignore requests feature and the once
record mode
- 1.0.0: *BACKWARDS INCOMPATIBLE*: Please see the 'upgrade' section in
the README. Take a look at the matcher section as well, you might
want to update your ``match_on`` settings. Add support for filtering
sensitive data from requests, matching query strings after the order
changes and improving the built-in matchers, (thanks to @mshytikov),
support for ignoring requests to certain hosts, bump supported
Python3 version to 3.4, fix some bugs with Boto support (thanks
@marusich), fix error with URL field capitalization in README (thanks
@simon-weber), added some log messages to help with debugging, added
``all_played`` property on cassette (thanks @mshytikov)
- 0.7.0: VCR.py now supports Python 3! (thanks @asundg) Also I
refactored the stub connections quite a bit to add support for the
putrequest and putheader calls. This version also adds support for
httplib2 (thanks @nilp0inter). I have added a couple tests for boto
since it is an http client in its own right. Finally, this version
includes a fix for a bug where requests wasn't being patched properly
(thanks @msabramo).
- 0.6.0: Store response headers as a list since a HTTP response can
have the same header twice (happens with set-cookie sometimes). This
has the added benefit of preserving the order of headers. Thanks
@smallcode for the bug report leading to this change. I have made an
effort to ensure backwards compatibility with the old cassettes'
header storage mechanism, but if you want to upgrade to the new
header storage, you should delete your cassettes and re-record them.
Also this release adds better error messages (thanks @msabramo) and
adds support for using VCR as a decorator (thanks @smallcode for the
motivation)
- 0.5.0: Change the ``response_of`` method to ``responses_of`` since
cassettes can now contain more than one response for a request. Since
this changes the API, I'm bumping the version. Also includes 2
bugfixes: a better error message when attempting to overwrite a
cassette file, and a fix for a bug with requests sessions (thanks
@msabramo)
- 0.4.0: Change default request recording behavior for multiple
requests. If you make the same request multiple times to the same
URL, the response might be different each time (maybe the response
has a timestamp in it or something), so this will make the same
request multiple times and save them all. Then, when you are
replaying the cassette, the responses will be played back in the same
order in which they were received. If you were making multiple
requests to the same URL in a cassette before version 0.4.0, you
might need to regenerate your cassette files. Also, removes support
for the cassette.play\_count counter API, since individual requests
aren't unique anymore. A cassette might contain the same request
several times. Also removes secure overwrite feature since that was
breaking overwriting files in Windows, and fixes a bug preventing
request's automatic body decompression from working.
- 0.3.5: Fix compatibility with requests 2.x
- 0.3.4: Bugfix: close file before renaming it. This fixes an issue on
Windows. Thanks @smallcode for the fix.
- 0.3.3: Bugfix for error message when an unreigstered custom matcher
was used
- 0.3.2: Fix issue with new config syntax and the ``match_on``
parameter. Thanks, @chromy!
- 0.3.1: Fix issue causing full paths to be sent on the HTTP request
line.
- 0.3.0: *Backwards incompatible release* - Added support for record
modes, and changed the default recording behavior to the "once"
record mode. Please see the documentation on record modes for more.
Added support for custom request matching, and changed the default
request matching behavior to match only on the URL and method. Also,
improved the httplib mocking to add support for the
``HTTPConnection.send()`` method. This means that requests won't
actually be sent until the response is read, since I need to record
the entire request in order to match up the appropriate response. I
don't think this should cause any issues unless you are sending
requests without ever loading the response (which none of the
standard httplib wrappers do, as far as I know. Thanks to @fatuhoku
for some of the ideas and the motivation behind this release.
- 0.2.1: Fixed missing modules in setup.py
- 0.2.0: Added configuration API, which lets you configure some
settings on VCR (see the README). Also, VCR no longer saves cassettes
if they haven't changed at all and supports JSON as well as YAML
(thanks @sirpengi). Added amazing new skeumorphic logo, thanks
@hairarrow.
- 0.1.0: *backwards incompatible release - delete your old cassette
files*: This release adds the ability to access the cassette to make
assertions on it, as well as a major code refactor thanks to
@dlecocq. It also fixes a couple longstanding bugs with redirects and
HTTPS. [#3 and #4]
- 0.0.4: If you have libyaml installed, vcrpy will use the c bindings
instead. Speed up your tests! Thanks @dlecocq
- 0.0.3: Add support for requests 1.2.3. Support for older versions of
requests dropped (thanks @vitormazzi and @bryanhelmig)
- 0.0.2: Add support for requests / urllib3
- 0.0.1: Initial Release
License
=======
This library uses the MIT license. See `LICENSE.txt <LICENSE.txt>`__ for
more details
.. |Build Status| image:: https://secure.travis-ci.org/kevin1024/vcrpy.png?branch=master
:target: http://travis-ci.org/kevin1024/vcrpy
.. |Stories in Ready| image:: https://badge.waffle.io/kevin1024/vcrpy.png?label=ready&title=Ready
:target: https://waffle.io/kevin1024/vcrpy

2
setup.cfg Normal file
View File

@@ -0,0 +1,2 @@
[bdist_wheel]
universal=1

View File

@@ -4,6 +4,7 @@ import sys
from setuptools import setup, find_packages
from setuptools.command.test import test as TestCommand
long_description = open('README.rst', 'r').read()
class PyTest(TestCommand):
@@ -18,19 +19,25 @@ class PyTest(TestCommand):
errno = pytest.main(self.test_args)
sys.exit(errno)
setup(
name='vcrpy',
version='1.4.2',
version='1.6.0',
description=(
"Automatically mock your HTTP interactions to simplify and "
"speed up testing"
),
long_description=long_description,
author='Kevin McCarthy',
author_email='me@kevinmccarthy.org',
url='https://github.com/kevin1024/vcrpy',
packages=find_packages(exclude=("tests*",)),
install_requires=['PyYAML', 'mock', 'six', 'contextlib2',
'wrapt', 'backport_collections'],
install_requires=['PyYAML', 'wrapt', 'six>=1.5'],
extras_require = {
':python_version in "2.4, 2.5, 2.6"':
['contextlib2', 'backport_collections', 'mock'],
':python_version in "2.7, 3.1, 3.2"': ['contextlib2', 'mock'],
},
license='MIT',
tests_require=['pytest', 'mock', 'pytest-localserver'],
cmdclass={'test': PyTest},

View File

@@ -4,6 +4,7 @@ from six.moves.urllib.request import urlopen, Request
from six.moves.urllib.parse import urlencode
from six.moves.urllib.error import HTTPError
import vcr
import json
def _request_with_auth(url, username, password):
@@ -66,6 +67,18 @@ def test_filter_post_data(tmpdir):
assert b'id=secret' not in cass.requests[0].body
def test_filter_json_post_data(tmpdir):
data = json.dumps({'id': 'secret', 'foo': 'bar'}).encode('utf-8')
request = Request('http://httpbin.org/post', data=data)
request.add_header('Content-Type', 'application/json')
cass_file = str(tmpdir.join('filter_jpd.yaml'))
with vcr.use_cassette(cass_file, filter_post_data_parameters=['id']):
urlopen(request)
with vcr.use_cassette(cass_file, filter_post_data_parameters=['id']) as cass:
assert b'"id": "secret"' not in cass.requests[0].body
def test_filter_callback(tmpdir):
url = 'http://httpbin.org/get'
cass_file = str(tmpdir.join('basic_auth_filter.yaml'))

View File

@@ -217,3 +217,20 @@ def test_post_file(tmpdir, scheme):
with open('tox.ini', 'rb') as f:
new_response = requests.post(url, f).content
assert original_response == new_response
def test_filter_post_params(tmpdir, scheme):
'''
This tests the issue in https://github.com/kevin1024/vcrpy/issues/158
Ensure that a post request made through requests can still be filtered.
with vcr.use_cassette(cass_file, filter_post_data_parameters=['id']) as cass:
assert b'id=secret' not in cass.requests[0].body
'''
url = scheme + '://httpbin.org/post'
cass_loc = str(tmpdir.join('filter_post_params.yaml'))
with vcr.use_cassette(cass_loc, filter_post_data_parameters=['key']) as cass:
requests.post(url, data={'key': 'value'})
with vcr.use_cassette(cass_loc, filter_post_data_parameters=['key']) as cass:
assert b'key=value' not in cass.requests[0].body

View File

@@ -0,0 +1,205 @@
# -*- coding: utf-8 -*-
'''Test requests' interaction with vcr'''
import json
import pytest
import vcr
from assertions import assert_cassette_empty, assert_is_json
http = pytest.importorskip("tornado.httpclient")
@pytest.fixture(params=['simple', 'curl', 'default'])
def get_client(request):
if request.param == 'simple':
from tornado import simple_httpclient as simple
return (lambda: simple.SimpleAsyncHTTPClient())
elif request.param == 'curl':
curl = pytest.importorskip("tornado.curl_httpclient")
return (lambda: curl.CurlAsyncHTTPClient())
else:
return (lambda: http.AsyncHTTPClient())
def get(client, url, **kwargs):
raise_error = kwargs.pop('raise_error', True)
return client.fetch(
http.HTTPRequest(url, method='GET', **kwargs),
raise_error=raise_error,
)
def post(client, url, data=None, **kwargs):
if data:
kwargs['body'] = json.dumps(data)
return client.fetch(http.HTTPRequest(url, method='POST', **kwargs))
@pytest.fixture(params=["https", "http"])
def scheme(request):
'''Fixture that returns both http and https.'''
return request.param
@pytest.mark.gen_test
def test_status_code(get_client, scheme, tmpdir):
'''Ensure that we can read the status code'''
url = scheme + '://httpbin.org/'
with vcr.use_cassette(str(tmpdir.join('atts.yaml'))):
status_code = (yield get(get_client(), url)).code
with vcr.use_cassette(str(tmpdir.join('atts.yaml'))) as cass:
assert status_code == (yield get(get_client(), url)).code
assert 1 == cass.play_count
@pytest.mark.gen_test
def test_headers(get_client, scheme, tmpdir):
'''Ensure that we can read the headers back'''
url = scheme + '://httpbin.org/'
with vcr.use_cassette(str(tmpdir.join('headers.yaml'))):
headers = (yield get(get_client(), url)).headers
with vcr.use_cassette(str(tmpdir.join('headers.yaml'))) as cass:
assert headers == (yield get(get_client(), url)).headers
assert 1 == cass.play_count
@pytest.mark.gen_test
def test_body(get_client, tmpdir, scheme):
'''Ensure the responses are all identical enough'''
url = scheme + '://httpbin.org/bytes/1024'
with vcr.use_cassette(str(tmpdir.join('body.yaml'))):
content = (yield get(get_client(), url)).body
with vcr.use_cassette(str(tmpdir.join('body.yaml'))) as cass:
assert content == (yield get(get_client(), url)).body
assert 1 == cass.play_count
@pytest.mark.gen_test
def test_auth(get_client, tmpdir, scheme):
'''Ensure that we can handle basic auth'''
auth = ('user', 'passwd')
url = scheme + '://httpbin.org/basic-auth/user/passwd'
with vcr.use_cassette(str(tmpdir.join('auth.yaml'))):
one = yield get(
get_client(), url, auth_username=auth[0], auth_password=auth[1]
)
with vcr.use_cassette(str(tmpdir.join('auth.yaml'))) as cass:
two = yield get(
get_client(), url, auth_username=auth[0], auth_password=auth[1]
)
assert one.body == two.body
assert one.code == two.code
assert 1 == cass.play_count
@pytest.mark.gen_test
def test_auth_failed(get_client, tmpdir, scheme):
'''Ensure that we can save failed auth statuses'''
auth = ('user', 'wrongwrongwrong')
url = scheme + '://httpbin.org/basic-auth/user/passwd'
with vcr.use_cassette(str(tmpdir.join('auth-failed.yaml'))) as cass:
# Ensure that this is empty to begin with
assert_cassette_empty(cass)
one = yield get(
get_client(),
url,
auth_username=auth[0],
auth_password=auth[1],
raise_error=False
)
with vcr.use_cassette(str(tmpdir.join('auth-failed.yaml'))) as cass:
two = yield get(
get_client(),
url,
auth_username=auth[0],
auth_password=auth[1],
raise_error=False
)
assert one.body == two.body
assert one.code == two.code == 401
assert 1 == cass.play_count
@pytest.mark.gen_test
def test_post(get_client, tmpdir, scheme):
'''Ensure that we can post and cache the results'''
data = {'key1': 'value1', 'key2': 'value2'}
url = scheme + '://httpbin.org/post'
with vcr.use_cassette(str(tmpdir.join('requests.yaml'))):
req1 = (yield post(get_client(), url, data)).body
with vcr.use_cassette(str(tmpdir.join('requests.yaml'))) as cass:
req2 = (yield post(get_client(), url, data)).body
assert req1 == req2
assert 1 == cass.play_count
@pytest.mark.gen_test
def test_redirects(get_client, tmpdir, scheme):
'''Ensure that we can handle redirects'''
url = scheme + '://httpbin.org/redirect-to?url=bytes/1024'
with vcr.use_cassette(str(tmpdir.join('requests.yaml'))):
content = (yield get(get_client(), url)).body
with vcr.use_cassette(str(tmpdir.join('requests.yaml'))) as cass:
assert content == (yield get(get_client(), url)).body
assert cass.play_count == 1
@pytest.mark.gen_test
def test_cross_scheme(get_client, tmpdir, scheme):
'''Ensure that requests between schemes are treated separately'''
# First fetch a url under http, and then again under https and then
# ensure that we haven't served anything out of cache, and we have two
# requests / response pairs in the cassette
with vcr.use_cassette(str(tmpdir.join('cross_scheme.yaml'))) as cass:
yield get(get_client(), 'https://httpbin.org/')
yield get(get_client(), 'http://httpbin.org/')
assert cass.play_count == 0
assert len(cass) == 2
# Then repeat the same requests and ensure both were replayed.
with vcr.use_cassette(str(tmpdir.join('cross_scheme.yaml'))) as cass:
yield get(get_client(), 'https://httpbin.org/')
yield get(get_client(), 'http://httpbin.org/')
assert cass.play_count == 2
@pytest.mark.gen_test
def test_gzip(get_client, tmpdir, scheme):
'''
Ensure that httpclient is able to automatically decompress the response
body
'''
url = scheme + '://httpbin.org/gzip'
with vcr.use_cassette(str(tmpdir.join('gzip.yaml'))):
response = yield get(get_client(), url, decompress_response=True)
assert_is_json(response.body)
with vcr.use_cassette(str(tmpdir.join('gzip.yaml'))) as cass:
response = yield get(get_client(), url, decompress_response=True)
assert_is_json(response.body)
assert 1 == cass.play_count
@pytest.mark.gen_test
def test_https_with_cert_validation_disabled(get_client, tmpdir):
cass_path = str(tmpdir.join('cert_validation_disabled.yaml'))
with vcr.use_cassette(cass_path):
yield get(get_client(), 'https://httpbin.org', validate_cert=False)
with vcr.use_cassette(cass_path) as cass:
yield get(get_client(), 'https://httpbin.org', validate_cert=False)
assert 1 == cass.play_count

View File

@@ -1,4 +1,6 @@
import pytest
from six.moves import xmlrpc_client
requests = pytest.importorskip("requests")
import vcr
@@ -72,3 +74,16 @@ def test_amazon_doctype(tmpdir):
with vcr.use_cassette(str(tmpdir.join('amz.yml'))):
r = requests.get('http://www.amazon.com')
assert 'html' in r.text
def test_xmlrpclib(tmpdir):
with vcr.use_cassette(str(tmpdir.join('xmlrpcvideo.yaml'))):
roundup_server = xmlrpc_client.ServerProxy('http://bugs.python.org/xmlrpc', allow_none=True)
original_schema = roundup_server.schema()
with vcr.use_cassette(str(tmpdir.join('xmlrpcvideo.yaml'))) as cassette:
roundup_server = xmlrpc_client.ServerProxy('http://bugs.python.org/xmlrpc', allow_none=True)
second_schema = roundup_server.schema()
assert original_schema == second_schema

View File

@@ -1,25 +1,25 @@
import copy
import inspect
import os
from six.moves import http_client as httplib
import contextlib2
import mock
import pytest
import yaml
from vcr.compat import mock, contextlib
from vcr.cassette import Cassette
from vcr.errors import UnhandledHTTPRequestError
from vcr.patch import force_reset
from vcr.stubs import VCRHTTPSConnection
def test_cassette_load(tmpdir):
a_file = tmpdir.join('test_cassette.yml')
a_file.write(yaml.dump({'interactions': [
{'request': {'body': '', 'uri': 'foo', 'method': 'GET', 'headers': {}},
'response': 'bar'}
]}))
a_cassette = Cassette.load(str(a_file))
a_cassette = Cassette.load(path=str(a_file))
assert len(a_cassette) == 1
@@ -87,33 +87,35 @@ def make_get_request():
@mock.patch('vcr.cassette.Cassette.can_play_response_for', return_value=True)
@mock.patch('vcr.stubs.VCRHTTPResponse')
def test_function_decorated_with_use_cassette_can_be_invoked_multiple_times(*args):
decorated_function = Cassette.use('test')(make_get_request)
for i in range(2):
decorated_function = Cassette.use(path='test')(make_get_request)
for i in range(4):
decorated_function()
def test_arg_getter_functionality():
arg_getter = mock.Mock(return_value=('test', {}))
arg_getter = mock.Mock(return_value={'path': 'test'})
context_decorator = Cassette.use_arg_getter(arg_getter)
with context_decorator as cassette:
assert cassette._path == 'test'
arg_getter.return_value = ('other', {})
arg_getter.return_value = {'path': 'other'}
with context_decorator as cassette:
assert cassette._path == 'other'
arg_getter.return_value = ('', {'filter_headers': ('header_name',)})
arg_getter.return_value = {'path': 'other', 'filter_headers': ('header_name',)}
@context_decorator
def function():
pass
with mock.patch.object(Cassette, 'load', return_value=mock.MagicMock(inject=False)) as cassette_load:
with mock.patch.object(
Cassette, 'load',
return_value=mock.MagicMock(inject=False)
) as cassette_load:
function()
cassette_load.assert_called_once_with(arg_getter.return_value[0],
**arg_getter.return_value[1])
cassette_load.assert_called_once_with(**arg_getter.return_value)
def test_cassette_not_all_played():
@@ -155,14 +157,14 @@ def test_nesting_cassette_context_managers(*args):
second_response = copy.deepcopy(first_response)
second_response['body']['string'] = b'second_response'
with contextlib2.ExitStack() as exit_stack:
first_cassette = exit_stack.enter_context(Cassette.use('test'))
with contextlib.ExitStack() as exit_stack:
first_cassette = exit_stack.enter_context(Cassette.use(path='test'))
exit_stack.enter_context(mock.patch.object(first_cassette, 'play_response',
return_value=first_response))
assert_get_response_body_is('first_response')
# Make sure a second cassette can supercede the first
with Cassette.use('test') as second_cassette:
with Cassette.use(path='test') as second_cassette:
with mock.patch.object(second_cassette, 'play_response', return_value=second_response):
assert_get_response_body_is('second_response')
@@ -172,12 +174,12 @@ def test_nesting_cassette_context_managers(*args):
def test_nesting_context_managers_by_checking_references_of_http_connection():
original = httplib.HTTPConnection
with Cassette.use('test'):
with Cassette.use(path='test'):
first_cassette_HTTPConnection = httplib.HTTPConnection
with Cassette.use('test'):
with Cassette.use(path='test'):
second_cassette_HTTPConnection = httplib.HTTPConnection
assert second_cassette_HTTPConnection is not first_cassette_HTTPConnection
with Cassette.use('test'):
with Cassette.use(path='test'):
assert httplib.HTTPConnection is not second_cassette_HTTPConnection
with force_reset():
assert httplib.HTTPConnection is original
@@ -188,12 +190,14 @@ def test_nesting_context_managers_by_checking_references_of_http_connection():
def test_custom_patchers():
class Test(object):
attribute = None
with Cassette.use('custom_patches', custom_patches=((Test, 'attribute', VCRHTTPSConnection),)):
with Cassette.use(path='custom_patches',
custom_patches=((Test, 'attribute', VCRHTTPSConnection),)):
assert issubclass(Test.attribute, VCRHTTPSConnection)
assert VCRHTTPSConnection is not Test.attribute
old_attribute = Test.attribute
with Cassette.use('custom_patches', custom_patches=((Test, 'attribute', VCRHTTPSConnection),)):
with Cassette.use(path='custom_patches',
custom_patches=((Test, 'attribute', VCRHTTPSConnection),)):
assert issubclass(Test.attribute, VCRHTTPSConnection)
assert VCRHTTPSConnection is not Test.attribute
assert Test.attribute is not old_attribute
@@ -201,3 +205,51 @@ def test_custom_patchers():
assert issubclass(Test.attribute, VCRHTTPSConnection)
assert VCRHTTPSConnection is not Test.attribute
assert Test.attribute is old_attribute
def test_decorated_functions_are_reentrant():
info = {"second": False}
original_conn = httplib.HTTPConnection
@Cassette.use(path='whatever', inject=True)
def test_function(cassette):
if info['second']:
assert httplib.HTTPConnection is not info['first_conn']
else:
info['first_conn'] = httplib.HTTPConnection
info['second'] = True
test_function()
assert httplib.HTTPConnection is info['first_conn']
test_function()
assert httplib.HTTPConnection is original_conn
def test_cassette_use_called_without_path_uses_function_to_generate_path():
@Cassette.use(inject=True)
def function_name(cassette):
assert cassette._path == 'function_name'
function_name()
def test_path_transformer_with_function_path():
path_transformer = lambda path: os.path.join('a', path)
@Cassette.use(inject=True, path_transformer=path_transformer)
def function_name(cassette):
assert cassette._path == os.path.join('a', 'function_name')
function_name()
def test_path_transformer_with_context_manager():
with Cassette.use(
path='b', path_transformer=lambda *args: 'a'
) as cassette:
assert cassette._path == 'a'
def test_func_path_generator():
def generator(function):
return os.path.join(os.path.dirname(inspect.getfile(function)),
function.__name__)
@Cassette.use(inject=True, func_path_generator=generator)
def function_name(cassette):
assert cassette._path == os.path.join(os.path.dirname(__file__), 'function_name')
function_name()

View File

@@ -4,6 +4,7 @@ from vcr.filters import (
remove_post_data_parameters
)
from vcr.request import Request
import json
def test_remove_headers():
@@ -67,3 +68,29 @@ def test_remove_nonexistent_post_data_parameters():
request = Request('POST', 'http://google.com', body, {})
remove_post_data_parameters(request, ['id'])
assert request.body == b''
def test_remove_json_post_data_parameters():
body = b'{"id": "secret", "foo": "bar", "baz": "qux"}'
request = Request('POST', 'http://google.com', body, {})
request.add_header('Content-Type', 'application/json')
remove_post_data_parameters(request, ['id'])
request_body_json = json.loads(request.body.decode('utf-8'))
expected_json = json.loads(b'{"foo": "bar", "baz": "qux"}'.decode('utf-8'))
assert request_body_json == expected_json
def test_remove_all_json_post_data_parameters():
body = b'{"id": "secret", "foo": "bar"}'
request = Request('POST', 'http://google.com', body, {})
request.add_header('Content-Type', 'application/json')
remove_post_data_parameters(request, ['id', 'foo'])
assert request.body == b'{}'
def test_remove_nonexistent_json_post_data_parameters():
body = b'{}'
request = Request('POST', 'http://google.com', body, {})
request.add_header('Content-Type', 'application/json')
remove_post_data_parameters(request, ['id'])
assert request.body == b'{}'

View File

@@ -1,6 +1,6 @@
import mock
import pytest
from vcr.compat import mock
from vcr.serialize import deserialize
from vcr.serializers import yamlserializer, jsonserializer

View File

@@ -1,7 +1,9 @@
import mock
import os
import pytest
from vcr import VCR, use_cassette
from vcr.compat import mock
from vcr.request import Request
from vcr.stubs import VCRHTTPSConnection
@@ -9,7 +11,10 @@ from vcr.stubs import VCRHTTPSConnection
def test_vcr_use_cassette():
record_mode = mock.Mock()
test_vcr = VCR(record_mode=record_mode)
with mock.patch('vcr.cassette.Cassette.load', return_value=mock.MagicMock(inject=False)) as mock_cassette_load:
with mock.patch(
'vcr.cassette.Cassette.load',
return_value=mock.MagicMock(inject=False)
) as mock_cassette_load:
@test_vcr.use_cassette('test')
def function():
pass
@@ -87,7 +92,10 @@ def test_custom_patchers():
assert issubclass(Test.attribute, VCRHTTPSConnection)
assert VCRHTTPSConnection is not Test.attribute
with test_vcr.use_cassette('custom_patches', custom_patches=((Test, 'attribute2', VCRHTTPSConnection),)):
with test_vcr.use_cassette(
'custom_patches',
custom_patches=((Test, 'attribute2', VCRHTTPSConnection),)
):
assert issubclass(Test.attribute, VCRHTTPSConnection)
assert VCRHTTPSConnection is not Test.attribute
assert Test.attribute is Test.attribute2
@@ -128,3 +136,75 @@ def test_with_current_defaults():
vcr.record_mode = 'all'
changing_defaults(assert_record_mode_all)
current_defaults(assert_record_mode_once)
def test_cassette_library_dir_with_decoration_and_no_explicit_path():
library_dir = '/libary_dir'
vcr = VCR(inject_cassette=True, cassette_library_dir=library_dir)
@vcr.use_cassette()
def function_name(cassette):
assert cassette._path == os.path.join(library_dir, 'function_name')
function_name()
def test_cassette_library_dir_with_decoration_and_explicit_path():
library_dir = '/libary_dir'
vcr = VCR(inject_cassette=True, cassette_library_dir=library_dir)
@vcr.use_cassette(path='custom_name')
def function_name(cassette):
assert cassette._path == os.path.join(library_dir, 'custom_name')
function_name()
def test_cassette_library_dir_with_decoration_and_super_explicit_path():
library_dir = '/libary_dir'
vcr = VCR(inject_cassette=True, cassette_library_dir=library_dir)
@vcr.use_cassette(path=os.path.join(library_dir, 'custom_name'))
def function_name(cassette):
assert cassette._path == os.path.join(library_dir, 'custom_name')
function_name()
def test_cassette_library_dir_with_path_transformer():
library_dir = '/libary_dir'
vcr = VCR(inject_cassette=True, cassette_library_dir=library_dir,
path_transformer=lambda path: path + '.json')
@vcr.use_cassette()
def function_name(cassette):
assert cassette._path == os.path.join(library_dir, 'function_name.json')
function_name()
def test_use_cassette_with_no_extra_invocation():
vcr = VCR(inject_cassette=True, cassette_library_dir='/')
@vcr.use_cassette
def function_name(cassette):
assert cassette._path == os.path.join('/', 'function_name')
function_name()
def test_path_transformer():
vcr = VCR(inject_cassette=True, cassette_library_dir='/',
path_transformer=lambda x: x + '_test')
@vcr.use_cassette
def function_name(cassette):
assert cassette._path == os.path.join('/', 'function_name_test')
function_name()
def test_cassette_name_generator_defaults_to_using_module_function_defined_in():
vcr = VCR(inject_cassette=True)
@vcr.use_cassette
def function_name(cassette):
assert cassette._path == os.path.join(os.path.dirname(__file__),
'function_name')
function_name()
def test_ensure_suffix():
vcr = VCR(inject_cassette=True, path_transformer=VCR.ensure_suffix('.yaml'))
@vcr.use_cassette
def function_name(cassette):
assert cassette._path == os.path.join(os.path.dirname(__file__),
'function_name.yaml')
function_name()

View File

@@ -1,5 +1,5 @@
[tox]
envlist = {py26,py27,py33,py34,pypy}-{requests27,requests26,requests25,requests24,requests23,requests22,requests1,httplib2,urllib317,urllib319,urllib3110,boto}
envlist = {py26,py27,py33,py34,pypy}-{requests27,requests26,requests25,requests24,requests23,requests22,requests1,httplib2,urllib317,urllib319,urllib3110,tornado,boto}
[testenv]
commands =
@@ -17,7 +17,7 @@ deps =
PyYAML
requests1: requests==1.2.3
requests27: requests==2.7.0
requests22: requests==2.6.0
requests26: requests==2.6.0
requests25: requests==2.5.0
requests24: requests==2.4.0
requests23: requests==2.3.0
@@ -26,4 +26,7 @@ deps =
urllib317: urllib3==1.7.1
urllib319: urllib3==1.9.1
urllib3110: urllib3==1.10.2
{py26,py27,py33,py34,pypy}-tornado: tornado
{py26,py27,py33,py34,pypy}-tornado: pytest-tornado
{py26,py27,py33,py34}-tornado: pycurl
boto: boto

View File

@@ -1,19 +1,18 @@
"""The container for recorded requests and responses"""
import functools
import logging
import contextlib2
import wrapt
try:
from collections import Counter
except ImportError:
from backport_collections import Counter
# Internal imports
from .compat import contextlib, collections
from .errors import UnhandledHTTPRequestError
from .matchers import requests_match, uri, method
from .patch import CassettePatcherBuilder
from .persist import load_cassette, save_cassette
from .serializers import yamlserializer
from .matchers import requests_match, uri, method
from .errors import UnhandledHTTPRequestError
from .util import partition_dict
log = logging.getLogger(__name__)
@@ -29,9 +28,11 @@ class CassetteContextDecorator(object):
from interfering with another.
"""
_non_cassette_arguments = ('path_transformer', 'func_path_generator')
@classmethod
def from_args(cls, cassette_class, path, **kwargs):
return cls(cassette_class, lambda: (path, kwargs))
def from_args(cls, cassette_class, **kwargs):
return cls(cassette_class, lambda: dict(kwargs))
def __init__(self, cls, args_getter):
self.cls = cls
@@ -39,7 +40,7 @@ class CassetteContextDecorator(object):
self.__finish = None
def _patch_generator(self, cassette):
with contextlib2.ExitStack() as exit_stack:
with contextlib.ExitStack() as exit_stack:
for patcher in CassettePatcherBuilder(cassette).build():
exit_stack.enter_context(patcher)
log.debug('Entered context for cassette at {0}.'.format(cassette._path))
@@ -49,10 +50,29 @@ class CassetteContextDecorator(object):
# somewhere else.
cassette._save()
@classmethod
def key_predicate(cls, key, value):
return key in cls._non_cassette_arguments
@classmethod
def _split_keys(cls, kwargs):
return partition_dict(cls.key_predicate, kwargs)
def __enter__(self):
# This assertion is here to prevent the dangerous behavior
# that would result from forgetting about a __finish before
# completing it.
# How might this condition be met? Here is an example:
# context_decorator = Cassette.use('whatever')
# with context_decorator:
# with context_decorator:
# pass
assert self.__finish is None, "Cassette already open."
path, kwargs = self._args_getter()
self.__finish = self._patch_generator(self.cls.load(path, **kwargs))
other_kwargs, cassette_kwargs = self._split_keys(self._args_getter())
if 'path_transformer' in other_kwargs:
transformer = other_kwargs['path_transformer']
cassette_kwargs['path'] = transformer(cassette_kwargs['path'])
self.__finish = self._patch_generator(self.cls.load(**cassette_kwargs))
return next(self.__finish)
def __exit__(self, *args):
@@ -61,20 +81,43 @@ class CassetteContextDecorator(object):
@wrapt.decorator
def __call__(self, function, instance, args, kwargs):
with self as cassette:
# This awkward cloning thing is done to ensure that decorated
# functions are reentrant. This is required for thread
# safety and the correct operation of recursive functions.
args_getter = self._build_args_getter_for_decorator(
function, self._args_getter
)
clone = type(self)(self.cls, args_getter)
with clone as cassette:
if cassette.inject:
return function(cassette, *args, **kwargs)
else:
return function(*args, **kwargs)
@staticmethod
def get_function_name(function):
return function.__name__
@classmethod
def _build_args_getter_for_decorator(cls, function, args_getter):
def new_args_getter():
kwargs = args_getter()
if 'path' not in kwargs:
name_generator = (kwargs.get('func_path_generator') or
cls.get_function_name)
path = name_generator(function)
kwargs['path'] = path
return kwargs
return new_args_getter
class Cassette(object):
"""A container for recorded requests and responses"""
@classmethod
def load(cls, path, **kwargs):
def load(cls, **kwargs):
"""Instantiate and load the cassette stored at the specified path."""
new_cassette = cls(path, **kwargs)
new_cassette = cls(**kwargs)
new_cassette._load()
return new_cassette
@@ -83,8 +126,8 @@ class Cassette(object):
return CassetteContextDecorator(cls, arg_getter)
@classmethod
def use(cls, *args, **kwargs):
return CassetteContextDecorator.from_args(cls, *args, **kwargs)
def use(cls, **kwargs):
return CassetteContextDecorator.from_args(cls, **kwargs)
def __init__(self, path, serializer=yamlserializer, record_mode='once',
match_on=(uri, method), before_record_request=None,
@@ -102,7 +145,7 @@ class Cassette(object):
# self.data is the list of (req, resp) tuples
self.data = []
self.play_counts = Counter()
self.play_counts = collections.Counter()
self.dirty = False
self.rewound = False

18
vcr/compat.py Normal file
View File

@@ -0,0 +1,18 @@
try:
from unittest import mock
except ImportError:
import mock
try:
import contextlib
except ImportError:
import contextlib2 as contextlib
else:
if not hasattr(contextlib, 'ExitStack'):
import contextlib2 as contextlib
import collections
if not hasattr(collections, 'Counter'):
import backport_collections as collections
__all__ = ['mock', 'contextlib', 'collections']

View File

@@ -1,23 +1,35 @@
import collections
import copy
import functools
import inspect
import os
import six
from .compat import collections
from .cassette import Cassette
from .serializers import yamlserializer, jsonserializer
from .util import compose
from . import matchers
from . import filters
class VCR(object):
def __init__(self, serializer='yaml', cassette_library_dir=None,
record_mode="once", filter_headers=(), ignore_localhost=False,
custom_patches=(), filter_query_parameters=(),
filter_post_data_parameters=(), before_record_request=None,
before_record_response=None, ignore_hosts=(),
@staticmethod
def ensure_suffix(suffix):
def ensure(path):
if not path.endswith(suffix):
return path + suffix
return path
return ensure
def __init__(self, path_transformer=lambda x: x, before_record_request=None,
custom_patches=(), filter_query_parameters=(), ignore_hosts=(),
record_mode="once", ignore_localhost=False, filter_headers=(),
before_record_response=None, filter_post_data_parameters=(),
match_on=('method', 'scheme', 'host', 'port', 'path', 'query'),
before_record=None, inject_cassette=False):
before_record=None, inject_cassette=False, serializer='yaml',
cassette_library_dir=None, func_path_generator=None):
self.serializer = serializer
self.match_on = match_on
self.cassette_library_dir = cassette_library_dir
@@ -46,6 +58,8 @@ class VCR(object):
self.ignore_hosts = ignore_hosts
self.ignore_localhost = ignore_localhost
self.inject_cassette = inject_cassette
self.path_transformer = path_transformer
self.func_path_generator = func_path_generator
self._custom_patches = tuple(custom_patches)
def _get_serializer(self, serializer_name):
@@ -69,27 +83,49 @@ class VCR(object):
)
return matchers
def use_cassette(self, path, with_current_defaults=False, **kwargs):
def use_cassette(self, path=None, **kwargs):
if path is not None and not isinstance(path, six.string_types):
function = path
# Assume this is an attempt to decorate a function
return self._use_cassette(**kwargs)(function)
return self._use_cassette(path=path, **kwargs)
def _use_cassette(self, with_current_defaults=False, **kwargs):
if with_current_defaults:
path, config = self.get_path_and_merged_config(path, **kwargs)
return Cassette.use(path, **config)
config = self.get_merged_config(**kwargs)
return Cassette.use(**config)
# This is made a function that evaluates every time a cassette
# is made so that changes that are made to this VCR instance
# that occur AFTER the `use_cassette` decorator is applied
# still affect subsequent calls to the decorated function.
args_getter = functools.partial(self.get_path_and_merged_config,
path, **kwargs)
args_getter = functools.partial(self.get_merged_config, **kwargs)
return Cassette.use_arg_getter(args_getter)
def get_path_and_merged_config(self, path, **kwargs):
def get_merged_config(self, **kwargs):
serializer_name = kwargs.get('serializer', self.serializer)
matcher_names = kwargs.get('match_on', self.match_on)
path_transformer = kwargs.get(
'path_transformer',
self.path_transformer
)
func_path_generator = kwargs.get(
'func_path_generator',
self.func_path_generator
)
cassette_library_dir = kwargs.get(
'cassette_library_dir',
self.cassette_library_dir
)
if cassette_library_dir:
path = os.path.join(cassette_library_dir, path)
def add_cassette_library_dir(path):
if not path.startswith(cassette_library_dir):
return os.path.join(cassette_library_dir, path)
return path
path_transformer = compose(add_cassette_library_dir, path_transformer)
elif not func_path_generator:
# If we don't have a library dir, use the functions
# location to build a full path for cassettes.
func_path_generator = self._build_path_from_func_using_module
merged_config = {
'serializer': self._get_serializer(serializer_name),
@@ -102,9 +138,14 @@ class VCR(object):
'custom_patches': self._custom_patches + kwargs.get(
'custom_patches', ()
),
'inject': kwargs.get('inject_cassette', self.inject_cassette)
'inject': kwargs.get('inject_cassette', self.inject_cassette),
'path_transformer': path_transformer,
'func_path_generator': func_path_generator
}
return path, merged_config
path = kwargs.get('path')
if path:
merged_config['path'] = path
return merged_config
def _build_before_record_response(self, options):
before_record_response = options.get(
@@ -185,6 +226,11 @@ class VCR(object):
return request
return filter_ignored_hosts
@staticmethod
def _build_path_from_func_using_module(function):
return os.path.join(os.path.dirname(inspect.getfile(function)),
function.__name__)
def register_serializer(self, name, serializer):
self.serializers[name] = serializer

View File

@@ -1,10 +1,9 @@
from six import BytesIO
from six import BytesIO, text_type
from six.moves.urllib.parse import urlparse, urlencode, urlunparse
try:
from collections import OrderedDict
except ImportError:
from backport_collections import OrderedDict
import copy
import json
from .compat import collections
def remove_headers(request, headers_to_remove):
@@ -31,8 +30,19 @@ def remove_query_parameters(request, query_parameters_to_remove):
def remove_post_data_parameters(request, post_data_parameters_to_remove):
if request.method == 'POST' and not isinstance(request.body, BytesIO):
post_data = OrderedDict()
for k, sep, v in [p.partition(b'=') for p in request.body.split(b'&')]:
if ('Content-Type' in request.headers and
request.headers['Content-Type'] == 'application/json'):
json_data = json.loads(request.body.decode('utf-8'))
for k in list(json_data.keys()):
if k in post_data_parameters_to_remove:
del json_data[k]
request.body = json.dumps(json_data).encode('utf-8')
else:
post_data = collections.OrderedDict()
if isinstance(request.body, text_type):
request.body = request.body.encode('utf-8')
for k, sep, v in (p.partition(b'=') for p in request.body.split(b'&')):
if k in post_data:
post_data[k].append(v)
elif len(k) > 0 and k.decode('utf-8') not in post_data_parameters_to_remove:

View File

@@ -2,9 +2,7 @@
import functools
import itertools
import contextlib2
import mock
from .compat import contextlib, mock
from .stubs import VCRHTTPConnection, VCRHTTPSConnection
from six.moves import http_client as httplib
@@ -54,6 +52,25 @@ else:
_CertValidatingHTTPSConnection = boto.https_connection.CertValidatingHTTPSConnection
# Try to save the original types for Tornado
try:
import tornado.httpclient
import tornado.simple_httpclient
except ImportError: # pragma: no cover
pass
else:
_AsyncHTTPClient = tornado.httpclient.AsyncHTTPClient
_SimpleAsyncHTTPClient = tornado.simple_httpclient.SimpleAsyncHTTPClient
try:
import tornado.curl_httpclient
except ImportError: # pragma: no cover
pass
else:
_CurlAsyncHTTPClient = tornado.curl_httpclient.CurlAsyncHTTPClient
class CassettePatcherBuilder(object):
def _build_patchers_from_mock_triples_decorator(function):
@@ -70,10 +87,11 @@ class CassettePatcherBuilder(object):
def build(self):
return itertools.chain(
self._httplib(), self._requests(), self._urllib3(), self._httplib2(),
self._boto(), self._build_patchers_from_mock_triples(
self._httplib(), self._requests(), self._urllib3(),
self._httplib2(), self._boto(), self._tornado(),
self._build_patchers_from_mock_triples(
self._cassette.custom_patches
)
),
)
def _build_patchers_from_mock_triples(self, mock_triples):
@@ -207,6 +225,27 @@ class CassettePatcherBuilder(object):
from .stubs.boto_stubs import VCRCertValidatingHTTPSConnection
yield cpool, 'CertValidatingHTTPSConnection', VCRCertValidatingHTTPSConnection
@_build_patchers_from_mock_triples_decorator
def _tornado(self):
try:
import tornado.httpclient as http
import tornado.simple_httpclient as simple
except ImportError: # pragma: no cover
pass
else:
from .stubs.tornado_stubs import VCRAsyncHTTPClient
from .stubs.tornado_stubs import VCRSimpleAsyncHTTPClient
yield http, 'AsyncHTTPClient', VCRAsyncHTTPClient
yield simple, 'SimpleAsyncHTTPClient', VCRSimpleAsyncHTTPClient
try:
import tornado.curl_httpclient as curl
except ImportError: # pragma: no cover
pass
else:
from .stubs.tornado_stubs import VCRCurlAsyncHTTPClient
yield curl, 'CurlAsyncHTTPClient', VCRCurlAsyncHTTPClient
def _urllib3_patchers(self, cpool, stubs):
http_connection_remover = ConnectionRemover(
self._get_cassette_subclass(stubs.VCRRequestsHTTPConnection)
@@ -322,10 +361,25 @@ def reset_patchers():
yield mock.patch.object(cpool, 'CertValidatingHTTPSConnection',
_CertValidatingHTTPSConnection)
try:
import tornado.httpclient as http
import tornado.simple_httpclient as simple
except ImportError: # pragma: no cover
pass
else:
yield mock.patch.object(http, 'AsyncHTTPClient', _AsyncHTTPClient)
yield mock.patch.object(simple, 'SimpleAsyncHTTPClient', _SimpleAsyncHTTPClient)
try:
import tornado.curl_httpclient as curl
except ImportError: # pragma: no cover
pass
else:
yield mock.patch.object(curl, 'CurlAsyncHTTPClient', _CurlAsyncHTTPClient)
@contextlib2.contextmanager
@contextlib.contextmanager
def force_reset():
with contextlib2.ExitStack() as exit_stack:
with contextlib.ExitStack() as exit_stack:
for patcher in reset_patchers():
exit_stack.enter_context(patcher)
yield

View File

@@ -204,13 +204,14 @@ class VCRConnection(object):
# no need to check that here.
self.real_connection.close()
def endheaders(self, *args, **kwargs):
def endheaders(self, message_body=None):
"""
Normally, this would atually send the request to the server.
Normally, this would actually send the request to the server.
We are not sending the request until getting the response,
so bypass this method for now.
so bypass this part and just append the message body, if any.
"""
pass
if message_body is not None:
self._vcr_request.body = message_body
def getresponse(self, _=False, **kwargs):
'''Retrieve the response'''

147
vcr/stubs/tornado_stubs.py Normal file
View File

@@ -0,0 +1,147 @@
'''Stubs for tornado HTTP clients'''
from __future__ import absolute_import
from six import BytesIO
from tornado import httputil
from tornado.httpclient import AsyncHTTPClient
from tornado.httpclient import HTTPResponse
from tornado.simple_httpclient import SimpleAsyncHTTPClient
from vcr.errors import CannotOverwriteExistingCassetteException
from vcr.request import Request
class _VCRAsyncClient(object):
cassette = None
def __new__(cls, *args, **kwargs):
from vcr.patch import force_reset
with force_reset():
return super(_VCRAsyncClient, cls).__new__(cls, *args, **kwargs)
def initialize(self, *args, **kwargs):
from vcr.patch import force_reset
with force_reset():
self.real_client = self._baseclass(*args, **kwargs)
@property
def io_loop(self):
return self.real_client.io_loop
@property
def _closed(self):
return self.real_client._closed
@property
def defaults(self):
return self.real_client.defaults
def close(self):
from vcr.patch import force_reset
with force_reset():
self.real_client.close()
def fetch_impl(self, request, callback):
headers = dict(request.headers)
if request.user_agent:
headers.setdefault('User-Agent', request.user_agent)
# TODO body_producer, header_callback, and streaming_callback are not
# yet supported.
unsupported_call = (
request.body_producer is not None or
request.header_callback is not None or
request.streaming_callback is not None
)
if unsupported_call:
response = HTTPResponse(
request,
599,
error=Exception(
"The request (%s) uses AsyncHTTPClient functionality "
"that is not yet supported by VCR.py. Please make the "
"request outside a VCR.py context." % repr(request)
),
)
vcr_request = Request(
request.method,
request.url,
request.body,
headers,
)
if self.cassette.can_play_response_for(vcr_request):
vcr_response = self.cassette.play_response(vcr_request)
headers = httputil.HTTPHeaders()
recorded_headers = vcr_response['headers']
if isinstance(recorded_headers, dict):
recorded_headers = recorded_headers.items()
for k, vs in recorded_headers:
for v in vs:
headers.add(k, v)
response = HTTPResponse(
request,
code=vcr_response['status']['code'],
reason=vcr_response['status']['message'],
headers=headers,
buffer=BytesIO(vcr_response['body']['string']),
)
callback(response)
else:
if self.cassette.write_protected and self.cassette.filter_request(
vcr_request
):
response = HTTPResponse(
request,
599,
error=CannotOverwriteExistingCassetteException(
"No match for the request (%r) was found. "
"Can't overwrite existing cassette (%r) in "
"your current record mode (%r)."
% (vcr_request, self.cassette._path,
self.cassette.record_mode)
),
)
callback(response)
def new_callback(response):
headers = [
(k, response.headers.get_list(k))
for k in response.headers.keys()
]
vcr_response = {
'status': {
'code': response.code,
'message': response.reason,
},
'headers': headers,
'body': {'string': response.body},
}
self.cassette.append(vcr_request, vcr_response)
callback(response)
from vcr.patch import force_reset
with force_reset():
self.real_client.fetch_impl(request, new_callback)
class VCRAsyncHTTPClient(_VCRAsyncClient, AsyncHTTPClient):
_baseclass = AsyncHTTPClient
class VCRSimpleAsyncHTTPClient(_VCRAsyncClient, SimpleAsyncHTTPClient):
_baseclass = SimpleAsyncHTTPClient
try:
from tornado.curl_httpclient import CurlAsyncHTTPClient
except ImportError: # pragma: no cover
VCRCurlAsyncHTTPClient = None
else:
class VCRCurlAsyncHTTPClient(_VCRAsyncClient, CurlAsyncHTTPClient):
_baseclass = CurlAsyncHTTPClient

16
vcr/util.py Normal file
View File

@@ -0,0 +1,16 @@
def partition_dict(predicate, dictionary):
true_dict = {}
false_dict = {}
for key, value in dictionary.items():
this_dict = true_dict if predicate(key, value) else false_dict
this_dict[key] = value
return true_dict, false_dict
def compose(*functions):
def composed(incoming):
res = incoming
for function in functions[::-1]:
res = function(res)
return res
return composed