CHANGES
=======

1.2.1
-----

* Propose 1.2.1 release
* build universal wheels
* Use a more clear error message
* Speficy db engine while get schema revision

1.2.0
-----

* Propose 1.2.0 release
* Add support of exporting results to elasticsearch 6.x
* fix cryptography version
* Use tempfile to get temp directory instead of hardcode '/tmp'
* [ElasticSearch] Reduce usage of no-name-action index
* Improve error messages of validators
* Include new validator in CHANGELOG
* Fix usage of InvalidTaskException in the task engine
* switch documentation job to new PTI
* import zuul job settings from project-config
* Fix deployment.check api
* Add map\_keys validator for map task args
* [task] Improve validation
* [task] Move TaskConfig to separate module
* Launch unittest coverage test

1.1.0
-----

* Propose Rally 1.1.0 release
* Declare that we support py3.6 and py3.7
* Fix all py37 incompatible things
* Update our requirements.txt and u-c files
* Ensure that we support py34,py36,py37 envs
* Add rally env cleanup command
* Fix error in HTML report generation for python 3
* Update the new feature of \`rally task detailed\` into CHANGELOG
* [ci] Fix floating-bug in functional tests
* Add filter option into \`rally task detailed\`
* [trivial] Update task exporter
* Extend json exporter and fix elastic exporter
* Fix minor typo in Rally Plugins doc
* Invalid links in doc
* Update config sample file
* Remove redundant message about openstack plugins
* Add py36 testenv
* fix typos

1.0.0
-----

* Prepare to Rally 1.0.0 release
* [evil] Remove OpenStack related plugins
* [verifiers] Fix identification of testr/stestr
* [evil][ci] Preparing to remove all OpenStack in-tree plugins
* Fix typos replace a OpenStack instance with an OpenStack instance
* Amend the spelling error of a word
* [env] Fix validation existing platforms in py3
* Drop all qpid related explanations
* Remove hardcode to openstack from required\_platform validator
* [plugins] Simplify usage of plugins with "default" platform
* fix error url

0.12.1
------

* Add release notes for 0.12.1
* Load options before importing packages
* Trivial: update url to new url

0.12.0
------

* [docs] Add release notes for 0.12.0
* Ignore several hacking rule
* change the error\_code to 0-255
* [version] Import rally.common.db.schema when necessary
* Redesign ResourceTypes
* [plugins] load config options from external plugins
* [types] Remove redundant function
* [docs] fix invalid "rst" usage
* Trivial: Update pypi url to new url
* [docs] Use proper reference for openstack plugins
* Remove oslotest from coverage job
* Remove several test dependencies
* [verify] Ger rid of calling pip internals
* [env] fix wrong call for logging
* Fix obtaining env spec in case of old deployment format
* [cli] Use json.loads instead of yaml while importing results
* Fix 'rally task report --tasks'
* Speed up RPS runner test cases
* Remove unused clitutils.process\_keystone\_exc

0.11.2
------

* Propose release notes for 0.11.2
* Fix obtaining env\_uuid from deployment
* Print warning about deprecation OpenStack plugins
* [envs] Add --only-spec option to pring spec
* Deprecate rally devstack plugin
* [plugin] Do not load pyc files
* get rid of oslo.utils
* Update the CirrOS image to 0.4.0
* Remove oslo.utils from plugins
* Fix constant\_for\_duration stucking on exceptions
* [envs] Creating a spec from system environment variables
* Fix the date format for db migrations files
* Fix the name format for db migrations
* Use for samples testing separated job
* Rework novaclient floating ips actions
* Update links in README
* [db] Refactor usage of session and serialization
* Update incorrect URL and word typo
* [openstack] Remove nova hosts specific logic
* Fix exporting workload with n/a success\_rate to es
* Fix documents title format error

0.11.1
------

* Fix the version of websocket-client
* Fix title for Rally 0.11.1 release notes
* Release notes for Rally 0.11.1
* [ci] Return 4 as a magic number for coverage job
* [db][ci] Add basic job for checking migrations
* Fix cli job
* Neutron scenario to delete subnets for the same network concurrently
* Don't use sitepackages for cli target
* [verify] Add support for sTestr
* [db] Rework 7287df262dbc migration
* Refacotr DB layer (part 1)
* Fix support for k8s-client 4.0
* Remove rally.task.utils.check\_service\_status method
* Return stdout data after running the command:
* Add new Gnocchi scenarios

0.11.0
------

* [docs] Fix date of Rally 0.11.0 release
* Fix uc for PrettyTable
* Release notes for 0.11.0
* [envs] Implmenet Env CLI commands
* [testing] Refactor CLI tests and utils
* [cfg] Use stderr for logging by default
* [docs] Fix several deployment configs
* [env] Be more strict for version of format
* Update requirements
* [cfg] Introduce rally.common.cfg
* Add timestamp item for Failure tab on html report
* [env][openstack] Change format of info method
* port more things to use envmanager
* Fix default values in OpenStack platform
* Fix grammar error
* update documentation reference
* [openstack] Change identifiers for floating ips
* Dockerfile - trivial fixes
* [cli] Fix docstring for --creds param of rally db ensure
* Add release notes for 0.10.1
* [envs] Change the format of plaforms data of EnvManager
* Improve Env Manager Interface
* Modify grammatical errors
* modify keystone spelling errors
* Add Gnocchi status scenario
* [jsonschema] Require specifying \`additionalProperties\` explicitly
* setup roles before validating resource and cleanup just after it
* Ability to specify external gateway info for router while creating network context
* Handle resources with no name
* Endpoint\_type of private is not valid
* Change line separator to the unix one
* [cleanup] Fix identifiers for Mistral resources
* Add Gnocchi capabilities scenario
* Replace Chinese punctuation with English punctuation
* Update quick\_start step\_1
* Modify the link to secure and correct
* Migrate tox-self test
* Introduce Gnocchi metric service
* Use upper-constraints.txt in tox envs
* Update image-show command
* [envs] Remove credentials base
* Update install\_rally.sh all available options
* Correct link address in doc
* [unittests]Port to new atomic actions
* Replace DeploymentEngine with EnvManager
* [envs] Add OpenStack platform (part 2)
* fix misspelling of 'configuration'
* modify sphinx-doc links
* Update links in bindep.txt
* Add timeout value into TimeoutException message
* keystone spelling errors
* [install] Use constraints in install script
* [envs] Introduce Env Manager & Platforms (part 1)
* Ensure cleanup is performed correctly on roles
* Change unsecure link to use https
* Revert "[CI] Unblock cinder job"
* Remove redudant UUID method from migrations
* Refactor DB related exceptions
* Worker entity removed
* [CI] Unblock cinder job
* Use openstack-tox-functional
* [CI] Fix Cinder update volume type scenario & service
* Improve imports in rally.common.db.sqlalchemy.api
* Fix spelling in rally.plugin.plugin
* Fix the misspelling of "configuration" and "scenario"
* Can not create Gnocchi client due to deprecated session options
* Fix the misspelling of "migration"
* [ci] remove legacy things from install job
* Remove outdated samples about deployment
* Add global path as constant in envutils
* Make new report to support task file and remove old report
* Use atomic actions in cinder contexts
* Modify Bgpvpn scenarios to test true bgpvpn
* [task] include contexts execution results
* [atomic actions] Add reset method
* Matching the tenant\_id when removing default security\_group
* Add support for ssh source
* [docker] Fix configuration
* Simplify usage of docker container
* OSProfiler Intergration
* [docs] Update info about cores
* Add option arguments of scenarios of floating ip creation
* Do not use own function to find migration target
* Fix JSON generation when scenario has hooks
* [ci] Split legacy-rally-dsvm-cli job
* Make the error code diffrent between running error and sla error
* Update the deprecated config schema
* Fix test\_wait\_timeout unit test
* Zuul: add file extension to playbook path
* Use base as the parent job, not base-integration
* Fix trends in case of failures in contexts
* Add correct version for pyOpenSSL
* Do not use latest k8s client
* Change command return value
* Allow to pass list kwargs to image resource type
* [CLI] Improve Rally DB UX
* [ci] Start migrating to zuul v3
* Add render\_complete\_data of chart
* [CLI] Add rally db ensure command
* [requirements] Stop using upper limits for all cases
* Use assertRegex instead of assertRegexpMatches
* Fix showing deployment config in case of old format
* [cli] Suppress PyMySQL warnings
* Fix a elasticsearch request's bug

0.10.0
------

* Propose release notes for Rally 0.10.0
* Add deployment name and uuid to ElasticExporter
* Remove trialing space from docs to fix py27 job
* [task] Rework junit-xml exporter
* Propose release notes for 0.9.2
* [docs] Fix displaying task exporters plugins ref
* Add release notes for 0.9.1
* Fix the datetime format of created\_at which was created by \`task results\`
* Add ElasticSearch exporter plugin
* Split task-export plugins by the proper modules
* import new json report into rally db
* [json] Several small improvements
* Clean docstrings of CLI methods from parameters
* [ci] Launch tox at real requirements
* Create new json exporter
* Fix trend reports
* Store which atomic fails
* [db] Fix a query for tasks filtering by tags
* Remove sample plugins
* Add task for getting OpenStack key performance metrics
* [functional] Fix hooks failures
* Remove AtomicMerger which is unused
* Fix certification task
* Fix the atomic action table of task results
* Finish work on renaming namespace to platform
* Specify platform in default contexts of GlanceImages.create\_and\_download\_image
* [functional] Fix failures in DeploymentTestCase
* [functional] Fix wrong way to obtaine the task uuid
* [functional] Fix trivial failures in TaskTestCase
* [ci] Start fixing CLI job
* update wait\_for to wait\_for\_status
* [opts] Register options with loading plugins
* [glance] mesure upload data to the image
* Add glance.DownloadImageData scenario
* [logging] Sort logging variables
* Update Rally config sample
* Unifies plugin configuration groups
* Rename config section of benchmark & tempest to openstack
* VMTasks.dd\_load: No random input for write test
* Deprecate install\_test.sh script
* Deprecate NovaImages.list\_images scenario
* Allow to specify resource name pattern via rally.conf
* Port task exporters under ValidatablePluginMixin
* [task results]Add some validation for importing task results
* Revert "[CI] Unblock CI by introducing Nova VMs cleanup bug"
* Use rally.plugins.openstack.osclients import everywhere
* Move rally.osclients to rally.plugins.openstack.osclients
* Remove oslo.serialization dependency
* Remove the direct dependency to oslo.i18n
* [logging] Check the right usage of logging.exception
* Save task's title and description
* Port inner stuff to the new task format
* [heat] Add "ERROR" status in wait\_for
* Improve Rally Logging (part 4)
* Port hooks to the new format
* Improve Rally Logging (part 3)
* Improve Rally Logging (part 2)
* [CI] Unblock CI by introducing Nova VMs cleanup bug
* Add a comment for a workload in rally-neutron.yaml
* Fix docstring of VMTasks.dd\_load\_test scenario
* Deprecate 'command' argument of VMTasks.dd\_load\_test
* Wrap several actions in VMTask with atomic timer
* Bump os-faults version to 0.1.15
* Use html-static instead of html\_static when exporting report from one file
* Make Cinder Job more stable (part 2)
* Fix the problem of switching hooks format
* [ci] Make cinder job more stable
* Improve Rally Logging (part 1)
* Fix Rally reporting in case if additive and complete keys are missing
* Fix auth issue in verification
* create a new server in each iteration when making snapshots from a server
* [ci] Fix path to a script to execute in hook
* [ci] Temporary workaround. Unblock gates
* Fix image list atomic action for both v1 and v2
* Fix duplicated hints printed in task detailed
* Fix api\_versions@openstack context plugin
* [validators] Port validators to use context object
* Improve context manager setup()/cleanup() logging
* Fix warning in scenario configure
* Use the right class name for MultiplePluginsFound
* [validators] Unify the order of arguments
* Rename arguments of workbook\_contains\_workflow validator
* use exception in validators
* Use full context names in scenario context configuration
* [plugins] Introduce <name>@<platform> syntax
* Fix Openstack.dummy plugin implementation
* Use "anyOf" instead of "oneOf" in users@openstack plugin
* Rename certification/ to tasks/
* Temporary fix for new ansible release
* [cli] Fix rally task import
* Remove dead code from task.types
* Simplify & Improve Dockerfile
* Fix formatting of RPS runner logging
* [task result]Fix a critical bug of api import\_result
* [ci] Update magnum, manila, mistral and monasca jobs to task v2
* [ci] Make rally-keystone-api-v2.yaml link to rally.yaml
* [ci] Update nova.yml to use task v2 format
* [ci] Update rally-designate.yml and rally-ironic.yml to use task v2 format
* Fix listing images in Glance V2
* [CLI] Add support for emtpy deployment
* [CI] Update the magnum image name & increase timeouts
* [ci] Update rally-heat.yml to use task v2 format
* [unit] Fix the warnings in unit task tests
* Fix "-" and "\_" samples check
* Add migration for workload stats
* Fix an issue with missing idle\_duration in reports
* Leave only one source of truth for atomic durations
* Add support for nested atomics for MainStatsTable
* Extend dummy\_random\_fail\_in\_atomic scenario
* Update requirements
* Make self-rally more stable
* Code optimization for detach volume
* Reword "benchmark" word in code base
* Switch Cinder Input Task to v2 format
* [db] Upsize title in task and subtask
* Update network wrapper to handle neutron plugins router handling
* [CI] Move to v2 format: senlin, watcher, zaqar
* Replace DbMigrationError with DBMigrationError
* Use new validators in Manila shares
* Finalize task format v2
* Move all defaults to TaskConfig
* Add a single workload subtask
* Fix to use "." to source script files
* [samples] Speed up checking samples
* Switch everything to \`rally db\`
* [CLI] Introduce rally db <command>
* Make task format v2 more user-friendly
* Fix Glance v1 update call
* Stop reinventing the is\_public filter for Glance v1
* Update requirments
* Update versions of python clients from global requirments
* Move old validators to the new style
* Fix message for required\_params validator
* [task schema]Move TASK\_RESULT\_SCHEMA and rename to OLD\_TASK\_RESULT\_SCHEMA
* modify some misspellings
* switch type(output) to isinstance in charts.validate\_output
* [task schema]Remove the unused schema TASK\_EXTENDED\_RESULT\_SCHEMA
* Add NovaServerGroups.create\_and\_delete\_server\_group
* Remove rally\_endpoint in api
* Simplify deployment config format
* Add analog of collections.namedtuple
* Pring task keys in more user-friendly way
* [task engine] Remove redundant classes
* Handle log message interpolation by the logger in common/
* [install]Remove BASE\_PIP\_URL which was used to pass pypi resource
* Remove redundant parent atomic action
* [atomic]Use \`while\` instead of recursion
* Fix cleaning action plan resources on watcher
* Add neutron.CreateAndDeleteSecurityGroupRule scenario
* Update and replace http with https for doc links in rally
* mofify gate reports to use static html because of some unreachable links
* Add GlanceImages.create\_and\_deactivate\_image
* [sla] Port sla mechanism to new atomic formats
* Improve default meta implementation
* Allow passing image properties to image creation scenarios
* [db] Fix serializer to work with ordered data
* [db] Optimize a db migration
* Fix displaying SLA in html report
* Add cinder.CreatQosAssociateAndDisassociateType
* Reword samples and setup.cfg
* Do not pass enable\_snat if ext-gw-mode extension is disabled
* Use prpoper wording instead of benchmark in rally-jobs
* Rephrase docs call things properly
* Remove redundant cleanup
* Add sla section to samples
* [ci]move unexpected resources table to console html
* Fixed order of arguments in assertEqual
* [ci] Rework launching cli job
* [cli] Add command to display db connection str
* Remove isfile checks on task and task args files
* Print the messages what can be traced back to real address
* Remove obsolate conepts
* Reduce 2 minutes of sleeps in functional cli task trends test
* Remove redudant configuration hacking in func samples test
* Ceilometer Samples Context should wait until samples are processed
* Fix step number in tutorial section in rally doc
* Remove dead code in task.validation module
* Fixed missing hyperlink for Ref[4] in osprofiler.rst in Rally Doc
* Update the documentation link for doc migration (additional)
* Fix Designate Plugins
* Reduce load in Nova and Cinder CI jobs
* Reduce load in gates in long running jobs
* Rename namespace -> platform in scenarios
* [db] Fix an issue with writting inf value
* Remove namespace in osclient module
* Rename namespace -> platform in context & validators plugins
* [db] Do not load unused fields
* [ci] Use not the latest xdist package
* Move tests.unit.test\_logging to tests.unit.common.test\_logging
* Improve deprecation warnings
* Add nova.BootServerAttachVolumeAndListAttachemnt
* Fix assertRaise cli output to only generate one file per test
* Add allow/deny access scenario to Manila
* Move os\_faults import from the root of module
* [small change]Change the format of plugin information
* Put db requirements to extra requirements
* Improve Task v2 format
* Rename namespace -> platform (part 1)
* [ci] do not fail on glance resources
* remove runtime dep on sphinx
* Fix an issue of database migration
* Added user creds in 'rally deployment check' err debug msg with tests
* Improve tox -e cli write to report failed commands as well
* Remove existing\_users context
* [db] Move serialize wrapper in the proper place
* Expand user path in Murano plugins
* Remove Cinder create and delete volume v1 API
* Update the documentation links for doc migration
* [cli] "rally task list" improvements
* Update URL home-page in documents according to document migration
* [ci] Turn off glance scenarios
* [Install] Fix postgresql installation
* Fix the bug of exporting report
* Fix junit-xml reporter always reporting failure
* Extend volume\_type\_exists with nullable option
* Fix an issue with None name of plugin
* Prepare for rally-openstack package
* [CI] Do not print logs when using JSON format
* Fix murano package scenarios
* Remove old way to describe scenario plugin via method
* Add NovaServers.boot\_and\_get\_console\_url
* Update .gitignore
* Get rid of extended\_results of task
* Store all data in db workload object
* add to judge whether resources end up with 'y' when clean up
* [ci] Experiment with check\_random\_request
* Fix floating bug with osprofiler test
* Update requirements
* [ci] Fail job in case if there are undeleted resources
* Add BGP VPN scenarios
* Fix misuse of assertIsNone
* 'File exists' validator and Utils class for validators
* [ci] Turn off failed cinder test
* Fixed shebang and permissions for scripts
* [docs] Fix representation of items
* fix router context to cleanup routers resources
* Switch to kubernetes upstream python client
* Fix enum validation in manila shrink and extend scenarios
* [ci]merge 3 patchs to unblock our ci
* [task] Simplify db layer
* Fix several TODOs in db layer
* Extend tags feature of tasks
* Fixup Rally Tutorial Page Link in README
* Refactor the related command of task report and export
* add check 'rally verify' related job in our CLI
* Add router in context
* Add manila extend/shrink scenarios
* [verify] Fix setting auth\_url
* Improve plugin sample and corresponding 'creating a plugin' tutorial
* Remove step\_9 tutorial from rally doc
* Add neutron.CreateAndShowSecurityGroupRule scenario
* fix Rally task [NovaServers.list\_servers] failed with network problems
* Add case\_insensitive parameter to enum validator
* deployment.list() return the serial result
* [docs] include required platforms of plugins
* cleanup old required\_services
* [ci] Test openstack deployment without admin
* make verification job great again
* Fix \`rally verify rerun\`
* remove redundant parameter in comments
* Deprecate CinderUtils and CinderWrapper
* cleanup bgp resources
* Add neutron.CreateAndListSecurityGroupRules scenario
* Allow deployment without admin creds
* Move 'volume\_type\_exists' scenario to '../validators.py'
* Deprecate optional\_action\_timer
* Fix an issue with namespace of scenario
* [raas] Adding a decorator of api\_wrapper
* [ci][samples] Reduce redundant authentications
* [ci][samples] Try to speed up checking samples
* [db] Fix porting task to the old format
* Revert "[ci] remove pytest-xdist dependency"
* Use manilaclient for API validation in Manila client
* Fix a typo and update a link address
* Remove usage of parameter enforce\_type
* Using fixtures instead of deprecated mockpatch module
* Various changes in custom\_image context
* [validator]cleanup required\_contexts warning
* Bugfix: fix the auth\_version
* Add CinderVolumeTypes.create\_volume\_type\_add\_and\_list\_type\_access
* [cleanup][network-wrapper] Handle auto-removed ports
* [cleanup] Fix flavors context
* [raas] the route need the api's version
* Move 'required-param-or-context' validator to '../validators.py'
* Revert "Temporary disable Cinder scenario that creating volume from image"
* OSprofiler support
* Allow CEPHFS as a valid share type in manila scenarios
* Move 'required\_contexts' validator to '../validators.py'
* Fix unit test of rally ci
* [validator]remove redundant validator and cleanup warning
* [validators] Cleanup warnings
* Move required-api-versions validator to '../validators.py'
* modify the annotations of delete\_snapshot
* Adds Networking-bgpvpn tests
* Add comment for 'seen' variable
* Moving of 'required\_cinder\_services' validator to '../validators.py'
* Temporary disable Cinder scenario that creating volume from image
* Add cinder.CreateAndSetQos scenario
* Replace kwargs with specific arguments
* Move 'restricted-parameters' validator to '../validators.py'
* [flavor]replace kwargs with details args
* [glance]delete redundant validation
* [Magnum] Context and scenario for Kubernetes
* Add cinder.CreateAndUpdateVolumeType scenario
* Replace assertRaisesRegexp with assertRaisesRegex
* Move 'validate\_heat\_template' validator to '../validators.py'
* Add a validator
* Refactor exceptions
* Move 'required\_services' validator to 'openstack/validators.py'
* [ci] Remove NovaSecGroup scenarios from rally-ci
* Modify log marker functions's usage in an exception
* update requirements
* [validators] Fix warning
* Fix loading plugins by entry-points
* Fix an issue with image interface of nova
* Fix allow\_ssh context
* Magnum: Move keypair handling at the cluster level
* Add task command \`rally task import\`
* Remove all calls to network interface via Nova
* Remove serverproviders & rarely used deployers
* Update requirements
* [plugins] Introduce plugins as python-packages
* Adds Networking-bgpvpn scenarios
* Add timestamp to atomic wrapper
* Add CinderVolumeTypes.create\_and\_update\_encryption\_type
* Add cinder.CreateAndListVolumeTypes scenario
* Add NeutronNetworks.create\_and\_show\_routers
* Move 'required\_clients' validator to '../validators.py'
* Move 'image-valid-on-flavor' validator to 'openstack/validators.py'
* Move 'required\_neutron\_extension' validator to '.../validators.py'
* Move 'external-network-exists' validator to 'openstack/validators.py'
* Fix identity v3 detection
* [image] Extract the common function
* Remove UnifiedOpenStackService
* [validation] Improve deprecation message
* [manila] Fix cleanup of manila\_share\_networks ctx
* Add CinderVolumeTypes.create\_get\_and\_delete\_encryption\_type
* Extend images@openstack context
* [cleanup] Add cinder.image\_volumes\_cache resource
* [cleanup] Introduce matcher for real names
* Adds Networking-bgpvpn tests
* fix list\_volumes in case image has no name
* [atomic\_actions]Port report to new atomic actions
* Add NovaServers.boot\_server\_and\_list\_interfaces
* Port context of volume type to block service
* [API] Abandon usage of classmethods & staticmethods in API
* [spec] OSProfiler integration in Rally
* [db] Add lazy-migration for atomics
* Fix details of nova\_utils
* Remove nova-networks
* Move 'image-exists' validator to 'openstack/validators.py'
* Unify QoSSpecs returned data
* Deprecate rally.osclients.EC2
* 'network-exists' validator removed
* Fix atomic wrapper
* Add support for namespaces into context manager
* Add cinder.CreateAndGetQos scenario
* fix type in name of EC2 service
* Fix the choice of default\_role
* Make openrc deployment file work with keystone v3
* Prepare tests to moving validators to 'openstack/validators.py'
* Fix doc generation for Python3
* [osclients] do not always strip auth\_url
* Split validation at 3 layers
* [validators] Port 'number' validator to plugin base
* [validation] Replace required\_openstack in neutron/network.py
* Add cinder.CreateAndListQos scenario
* GlanceImages.create\_and\_update\_image
* Extend required\_paramaters validator
* Revert "Unblock other projects"
* Fix devstack rally deployment config
* Unblock other projects
* [API] Remove the DeprecatedAPIClass
* Add nova.BootServerAndAttachInterface scenario
* Add neutron.CreateAndShowSecurityGroup scenario
* Replace 'assertEqual(True, a)' with 'assertTrue(a)' and add hacking rule
* [ci] print available images, volumes, nets
* Fix a typo
* Fixing of creating of images for rally verify
* Fix error when showing a failed deployment
* Add samples to context directory
* [API] Make return data of Task API serializable
* [core] Refactor validation
* [ci] add task file for telemetry job
* Remove OPTs from 'utils.py'
* Optimize the link address
* [ci] remove pytest-xdist dependency
* Fix task results bug
* Fix insecure and cacert warnings
* Fix list\_and\_get\_host scenario bug
* Fixed project creation with Keystone v3
* Add cinder qos and nova server groups in ci/osresources
* Add properties argument when create ironic node
* Add NeutronNetworks.set\_and\_clear\_router\_gateway
* Add GlanceImages.create\_and\_get\_image
* [validation] Use jsonschema validator in ScenarioRunner plugins
* [validation] Use jsonschema validator in Context plugins
* credentials is not updated after deployment is recreated
* [cleanup] Fix neutron quotas resource
* Propose a spec for Rally-as-a-Service
* Reduce hardcode to OpenStack
* [cleanup] Fix ec2 credential resource
* Switch glance wrapper to image service that cinder service calls
* [plugins] Fix ironic scenarios
* [validation] Use jsonschema validator in SLA plugins
* [Service] Update get\_image functions' input and add a param in UnifiedImage
* [validation] Use jsonschema validator in Hook and Trigger plugins
* [cleanup] Use workload uuid in resource names
* [validation] Replace required\_openstack in all scenarios
* [ci] Change image name for senlin job
* Add py files in doc directories to pep8 check
* Requirements file missing python-virtualenv
* [Service] Port all glanceWrapper to image\_service
* Use cinder service in cinder volumes context
* [cleanup] Restrict "cleanup" to scenario resources only
* [cleanup] Fix neutron resources
* [CI] Merge subtasks in self-rally.yaml
* [Core] Allow more than 1 workload in v2 task config
* [REPORT] Adjust started\_at param's form in task\_report
* [cleanup] Restrict "admin\_cleanup" to scenario resources only
* [CI] Fix typo in osresources.py
* Switch cinder scenarios to cinder service
* [Validator] Add validator plugin base
* [Service] Fix some details in image\_service
* [cleanup] Modify quotas resources
* [Install] Add --force-yes argument to apt-get
* Add  neutron.CreateAndShowSubnet scenario
* Corrected the grammar mistake
* [Service] Port all glance scenarios to Image Service
* [CI] Move default rally scenarios to rally-tox-self job
* [ci] Add features.yaml file
* [Core] Add support for default platform and rally self job
* Make return data of Deployment API serializable
* [verify] Fix \`rally verify rerun\`
* [CI] Show test names in job logs
* Fix MagnumClusters.create\_and\_list\_clusters
* refactor atomics actions
* [CI] Enable embedded css on py.test report
* Add "description" to task config
* [CI] Add tox self env that do nothing
* Avoid shell=True in subprocess
* Add NeutronNetworks.create\_and\_show\_ports scenario
* OSResources should be extended to cover all resorces
* modify the parameter error in comments
* Update deployment samples
* Fix deployment --fromenv option
* Adjusting introduction of gate
* Port CinderWrapper to service modules
* Fix nics format in servers ctx
* [Core] Create new plugin based class for credentials
* Make return data of Verify/Verification API serializable

0.9.0
-----

* add release notes for 0.9.0
* [docs] Update docs in jsonschema
* [requirements] Sync requirements
* [cleanup] Introduce NoName obj for res without name field
* [cleanup] Remove old nova\_client judge in NovaServer
* [docs] Extend verification reporters with output examples
* Revert "Neutron's floatingip add desc param"
* [verify] Extend rerun cmd with tag and concurrency args
* Enabling support of 'script\_inline' option
* [CI] Show all attributes in osresources check
* [verify] Extend rerun cmd with detailed and no-use args
* Neutron's floatingip add desc param
* Do not check Glance V1
* [CI] skip some resources while checking cleanup
* Modify variable's usage in Log Messages
* [CLEANUP] Add ability to specify superclass for cleanup
* Change volumes size or time of cind ci
* Add cinder backups quota support
* Add hacking rules for deprecated LOG.warn
* Add cinder.CreateAndGetVolumeType scenario
* Corrected the sentence in problem description
* [ci] Add status field to resource identifiers
* [img] changed the spelling of third party in the image
* [Test] Fix randomly failing unit test
* Add nova.CreateAndGetKeypair scenario
* Modifing of test on docstrings
* [FIX] Port some cleanup to admin\_cleanup in scenarios
* Add CinderVolumeTypes.create\_and\_delete\_encryption\_type
* Fix ordered dict rendering for task templates
* [Core] Make task engine more abstract
* Adjust auth\_url's deal in Keystone
* [CI] Update hacking and fix pep8 errors
* Add NovaServerGroups.create\_and\_get\_server\_group
* Refactoring of deployment credentials
* [DOC] Remove redundant info of 'rally show' in docs
* Update core members
* [Verify] Fix issue with verifier versions
* Remove Endpoint from Rally common objects
* Fix simple typo in file network.py
* [validation] validate scenario default\_context
* [CI] Fix neutron job
* removed extra 'can' from description
* [CI] Increase SLA values to improve stability of CI
* [CI] Fix image regexp and version
* Remove logging import unused
* Improve Task status list
* pprint jsonschema in plugin reference
* Fix used jsonschemas
* [Trivial]Fix some spelling errors
* [FIX] fix params in creating floating-ip
* Remove BootRuncommandDeleteCustomImage scenario
* [ci] Check all rally-jobs tasks
* Fix race-condition while making dirs in cli tests
* Switch deployment and verify to custom yaml
* [Trivial]Correct some type errors
* [TASK] Add created\_time at task-report-html
* [Trivial]Fix some type errors in ../README.rst
* [Trivial]Fix some type errors
* Typo fix: interupt => interrupt
* [Core] Check deployment status
* [Trivial]Fix type errors
* Switch task parser to custom yaml
* [Verify] Fix issue with test name regex in Testr plugin
* Add custom yaml class
* Add expected variable to heat.create\_client
* [docs] reduce sphinx warnings
* Move DD\_Load task from BASH script to Rally scenario
* Correct some annotation spelling mistakes
* [Verify] Refactoring of Tempest verifier plugin
* Add cinder.CreateAndSetVolumeType scenario
* Add keystone.CreateAndUpdateUser scenario
* Typo fix: splitted => split
* osclient: Pass endpoint\_type in kw\_args to client
* Correct some annotation errors
* [docs] Fix \_get\_info for function based scenarios
* [docs] Improve plugins and cli references
* Add keystone.CreateAndListAllRoles
* Use \_create\_network in \_create\_network\_structure
* Fix JUnit-XML report
* Fix backup create and restore scenario race problem
* add more badges
* Expand context validator logic
* Add support for enable\_plugin
* Fix Zaqar client auth

0.8.1
-----

* Release 0.8.1
* Fix novaclient requirement
* [doc] Add documentation for hook and trigger plugins

0.8.0
-----

* Use new network for each subnet scenario run
* [doc][verify] Some more changes in docs before release
* Typo fix: mentiones => mentioned
* [docs] several fixes before release
* [Verify] Add docs with common workflow for verifications
* [CLI] Allow changing deployment config on recreate
* Add nova.ResizeShutoffServer scenario
* Add quotas.CinderGet scenario
* Add release notes for 0.8.0
* [install bug]Fix directory changed when installing env
* Sync requirements
* [docs] Describe verification component
* [Verify] Add 'crashed' status to verifications
* [verification] Introduce rally verify show-verifier command
* Typo fix: binded => bound
* [docs][info] Introduce new method for custom plugin docs
* Add argument to test boot server with file
* [Verify] Add --detailed flag to \`rally verify start\` cmd
* [Verify] Add tags to verifications
* [verification] add junit reporter
* [verification][api] Store default repo to db
* [docs] Move several docs
* [docs] Several improvements for extensions
* Fix trailing space added at end of lines
* [Verify] The latest cleanup before new release
* [Deployment] Delete redudant printed info of deployment
* Add neutron lbaasv2 loadbalancer scenarios
* [Test] Add test for simple app based on rally api
* [Context] Add region\_name in credential
* [Verify] Delete HTMLStaticReporterTestCase class
* Use interface argument name for keystoneclient
* [common] create base class for verifier contexts
* [verification] Add HTML-Static reporter plugin
* Add CinderVolumeTypes.create\_and\_list\_encryption\_type
* Add cinder.CreateIncrementalVolumeBackup scenario
* [verification] Save timestamps of tests
* Replace yaml.load() with yaml.safe\_load()
* change neutron client's endpoint
* [task] Update the task log to make it consistent
* Removed redundant word: 'is'
* [Verify] Improve names of args for configure\_verifier method
* Explicitly pass endpoint type to Neutron client
* Fix hook and trigger validation
* Make glance v2 the default
* [docs][7] Re-design docs to cover all user-groups
* [docs][6] Re-design docs to cover all user-groups
* [plugins][cli] Include plugin base in plugins list
* Make verification reporter pluggable
* [verification] add force flag to configure verifier
* Switch admin\_cleanup to usercleanup
* [Verify] Small improvoments for \`rally verify show\` cmd
* [Verify] Introduce \`rally verify rerun\` cmd
* [Core] MetaMixin: improve implementation of default values
* [Core] Skip db check for plugin commands
* Get glance+rally to work under python3
* Support PYTHON3 gate jobs
* [Core] Show proper error message when db is missing
* Additional checks for ceilometers and designate scenarios
* Move junit module to rally.common.io
* Add NovaServerGroups.create\_and\_list\_server\_groups
* Fix a typo
* Removes unnecessary utf-8 encoding
* Add neutron.CreateAndShowNetwork scenario
* [verification] use vertical table
* [install] Add iputils to bindep
* [install] Fix install\_rally.sh
* [Verify] Add DB migration to fix test results for verifications
* [Verify] Small improvements for verification component
* Check verification component refactoring
* [verification] Fix small bugs in verification component
* Save raw task results in chunks
* [Verify] Fix small bugs in CLI interface
* [verification] Introduce Tempest verifier
* Add nova.CreateFlavorAndAddTenantAccess scenario
* [verification] Add CLI interface
* Change "a SQL" to "an SQL"
* Fix simple typos in file hypervisors.py
* [Core] Add database revision check to API initialization
* Remove cueclient
* [cli]remove deprecated \`rally show\` command
* [Verify] Improve messages in logging and exceptions for verify API
* [verification] add api
* [verification] Add base manager and context classes
* [Bindep]Use bindep lib to install system packages
* Fix a bug of wait\_for\_status when running timeout
* [verification] Introduce new objects
* Fix the link in rally-jobs
* [verification] Modify subunit\_v2 parser
* [verification] Introduce BackupHelper
* [verification] Prepare reports to new structure
* [API] Add API class
* [verification] Remove outdated stuff
* Replace six.iteritems with dict.items()
* Use assertGreater(len(x), y) instead of assertTrue(len(x) > y)
* Change \_ to \_LW for LOG.warning
* [Hooks][Reports] Make hook config visible in report
* [Hooks][Reports] Fix task report crash
* Use sa.Text instead of sa.String(1000) for verifier description
* [db] Migrate old verification results to new format
* [ci] Use neutron quotas for security groups
* Additional checks for nova scenarios
* [jobs] Test all keystone scenarios
* [Core] Remove map\_for\_scenario
* Add Subtask and Workload classes
* [core][constant\_runner] add validation for concurrency
* Port all keystone scenarios to Identity Service
* [Services] redesign metaclass to support subclasses
* [Spec] Updates for improve\_atomic\_actions\_format.rst
* Make our CI great again
* Use \_list\_hypervisors() method in nova aggregates
* Add Apache 2.0 license to source file
* [ci] stop checking verification component
* [Install] Remove redundant getpip variable
* Fix db.deployment\_update
* [Core][Bug] Fix 'task start --abort-on-sla-failure'
* Bump virtualenv version in install\_rally.sh
* [CI][cli] use env variable to determine KeystoneV3
* [docs][5] Re-design docs to cover all user-groups
* [docs][4] Re-design docs to cover all user-groups
* [ci] Fix our jobs
* [docs][3] Re-design docs to cover all user-groups
* [docs][2] Re-design docs to cover all user-groups
* CreateAggregateAddHostAndBootServer scenario for aggregates
* [CLI] group commands in help message
* Show team and repo badges on README
* update keypair's context
* Small correction to db\_refactoring.rst
* Fix rally verify ci test
* Refactor DB shema for task results
* [gates] Fix rally gates
* Remove extra field from watcher plugin
* [docs]add info about trello board and gitter chat
* [docs][1] Re-design docs to cover all user-groups
* [Rally profile] Increase sla rates for generate names
* Add CinderVolumeTypes.create\_volume\_type\_and\_encryption\_type
* Add an option to pass workflow input and parms to mistral execution
* Add ability to increase rps with certain duration
* [requirements] Sync requirements
* [Manila] Add scenarios for Manila share metadata
* Add CinderVolumes.create\_and\_accept\_transfer
* Remove hosts in a nova aggregate before deleting the aggregate
* [ci] add test hook for certificaiton task
* [cli] do not sort results of tasks
* Add NeutronNetworks.list\_agents
* [Rally profile] Performance of creating atomic timers
* Add cinder.CreateVolumeType scenario
* Allow create image params in glance scenario
* [Services] Introduce Identity Service
* Add CeilometerAlarms.create\_and\_get\_alarm
* [Verify][Reports] Small improvements of Rally verify report
* Add missing device\_owner for L3 HA's case
* [Manila] Fix CI job configuration
* Add nova resources in ci/osresources
* Fault injection hook plugin
* [Reports] Fix sorting for Hooks column in overview table
* Update wait\_for to wait\_for\_status
* Fix simple typos in file hypervisors.py
* [Services] Add base class Service
* Replace six iteration methods with standard ones
* Add nova.ListAndGetHosts scenario
* Fix a typo in 'rally verify compare' msg
* Add test and sample to CreateAndListNetworks
* Add nova.ListAndSearchHypervisors scenario
* Add Quotas.nova\_get The scenario list the quotas for nova
* Increase number of iterations in RallyProfile.generate\_names\_in\_atomic
* Use upper-constraints for tox envs
* Benchmark for generate\_random\_name
* Add cinder.ListAndGetVolume scenario
* Add CinderVolumes.create\_volume\_and\_update\_readonly\_flag
* Add CinderVolumes.list\_transfers
* Don't attempt to escalate rally-manage privileges
* Fix some typos in release note
* Add nova.CreateAndDeleteFlavor scenario
* Add CinderVolumes.list\_types
* Add keystone.CreateAndGetRole scenario
* Propose Oleg Bondarev to neutron-plugins-core
* [Manila] Add create\_and\_list\_share scenario for Manila
* [Trivial] Fix some typos in rally
* Avoid variables overwrite by keystone default config
* [CLI] Fix mentions of deprecated command \`rally verify compare'
* [Reports] Get rid of Mako in reports, finally
* Add nova.getHypervisorStatistics
* [Tests] Fix functional test\_time\_hook
* [Reports] Significant improvements in verification report
* Add v3 role create usage
* sys\_call hook: Open subprocces streams in preferred encoding
* Add NovaAggregates.create\_and\_get\_aggregate\_details
* Add nova.ListAndGetHypervisorUptime
* Add nova.BootServerAndAddSecgroup
* Add option to create mistral execution from workbook

0.7.0
-----

* [release] notes for 0.7.0
* Remove pycparser restriction from requirements
* Move 'hook\_section.rst' spec to doc/specs/implemented/hook\_plugins.rst
* [docs] Fix plugin references for class-based scenarios
* Sync requirements before release
* [Gates] Synchronize script instance\_test.sh
* Add NovaAggregates.create\_aggregate\_and\_add\_host
* [Tests] Improve RallyCliError exception
* Add nova.CreateFlavorAndSetKeys
* Add nova.ListAndGetHypervisors
* Add NovaAggregates.create\_and\_update\_aggregate
* [Plugins] Replace instance\_dd\_test.sh with instance\_test.sh
* [Verify] Fix \`rally verify installplugin\` cmd
* Moving of Nova.Servers scenarios into separate classes
* [Hooks][Reports] Show Hooks output in HTML report
* Port all Sahara scenarios to new style
* Moving of Neutrone scenarios into separate classes
* Add Keystone token validation
* [Verify] Fix tempest keystone v3 tests
* [Verify] Fixing issue with downloading images for the tests
* Moving of Nova scenarios into separate classes
* Update of docstrings to support common-style scenarios
* Port of Auth and Desicnate to new style
* Moving of EC2, Fuel and Glance scenarios into separate classes
* Add periodic trigger plugin
* Pin pycparser version to ==2.13
* Refactor hook stuff
* Add timeout for tests execution
* Add NovaAggregates.create\_and\_delete\_aggregate
* allow passing of specific volume type
* Replace warning.warn with LOG.warning
* Add Apache 2.0 license to source file
* Using sys.exit(main(sys.argv)) exit the program
* Update project info page
* Add BootServerAssociateAndDissociateFloatingIp
* [magnum] Rename to cluster and cluster\_template
* Docstrings should not start with a space
* Remove unnecessary \_\_init\_\_
* Remove admin\_domain\_name from Credetials
* Using sys.exit(main()) instead of main()
* Use assertEqual() instead of assertDictEqual()
* Add NovaAggregates.create\_and\_list\_aggregates
* Fix typos in release note
* [API] Allow to delete stopped tasks without force=True
* Update config sample file with latest config options
* Add NovaFlavors.create\_and\_get\_flavor
* [Magnum] Scenario to create a Magnum bay
* Use hook executer inside task engine
* Fix column name for output of \`rally task detailed --iterations-data\`
* Revert "[rally-jobs] unblock gates"
* Fixing docker-check jenkins job
* Add sys\_call hook
* Moving of Heat scenarios into separate classes
* Moving of Ironic and Keystone scenarios into separate classes
* [magnum] Move list baymodels in a scenario class
* Remove unnecessary setUp
* Adding the task attribute to the Hook instance
* [Magnum] Context and scenario for Magnum baymodel
* Moving of Volumes scenarios into separate classes
* Moving set of scenarios into separate classes
* New plugin type - Hook
* New plugin type - Trigger
* Moving of Ceilometer Alarms scenarios into separate classes
* [rally-jobs] unblock gates
* [task] Fix verification log
* [Reports] Fix layout in base.html
* [Tests] Add missed tests/unit/common/test\_version.py
* Fix scenario miscount in the tutorial
* Port Senlin and others scenarios to new style
* Rework optional-requirements
* Improve debug logging level in Engine
* Move clients to the usage of sessions
* Moving of Ceilometer scenarios into separate classes
* [CLI] Add version info to task start output
* [Tests] Ignore too long mock names
* Refactor sleep\_between
* Update error messages for ceilometer event scenarios
* [gatest] Comment several ceilometer scenarios
* [Manila] Add security-services context
* [requirements] Update requirements
* [Reports] Add new OutputTextArea chart plugin
* Fix the sample of scenario plugin
* [Plugins] Remove deprecated Dummy.dummy\_with\_scenario\_output
* Remove time.sleep

0.6.0
-----

* Add simple wrapper for pytest
* [osclients] Fix zaqar client
* [release] Rally 0.6.0
* Sync and add upper limit to requirements
* Bump pytest-html version to 1.10.0
* [senlin]Add context support for Senlin profile
* Add \_\_ne\_\_ built-in function
* [verify] Fix obtaining project id
* Add \`extra\` parameter to ExistingCloud deployment
* [Plugins] Rename workload dammy.failure -> Dummy.failure
* [osclients] use keystone auth\_ref directly
* [devstack] fix keystone v3 deployemnt config
* Fix some typos in hosts.py
* [Reports] Use timestamps on X axis in trends report
* Don't use config option sqlite\_db
* Fix neutron unstable job
* Moving of Quotas scenarios into separate classes
* Fix neutron gate jobs
* TrivialFix: Remove logging import unused
* Translate info-level log messages for LOG.error
* Remove Pavel Boldin from maintainers page
* Add watcher audit template context and 2 scenarios
* Edited the tempest verify docs
* Use a shortcut to configure function
* Fix certification task instructions
* Fix typos in the docs
* Remove useless \_\_init\_\_
* [sshutils] Catch socket.error in put\_file
* Use more specific asserts in tests
* [Reports] In charts, fix setting red color for non-error metric
* Put in more propriate place test results
* Allow volume\_type to be None since None is a valid value
* Update hook\_section spec
* [Verify] Adding 'add-options' arg to \`rally verify genconfig\` cmd
* Fix some typos in the files
* [Plugins] Add executable availability check to instance\_test.sh
* [Reports] Hide menu list if there is no data at all
* [Doc][Reports] New documentation chapter about HTML reports
* Correct spelling mistake in test case name
* Moving of Ceilometer Eventss scenarios into separate classes
* [spec] New Tasks Configuration section - hook
* Use attachment\_id instead of volume\_id when detaching volume
* [Reports] Fix trends report generation if there are n/a results
* Moving of CeilometerQueries scenarios into separate classes
* Fix ExistingCloud properties
* Moving of Dummy scenarios into separate classes
* Skip checking samples in coverage job
* Moving of HttpRequests scenarios into separate classes
* [Verify] Deprecating 'tests-file' and 'xfails-file' args
* Update requirements
* [Verify] Fixing issue with system-wide Tempest installation
* Add success rate chart to trends report
* Remove deprecated log module
* Add create\_audit\_template\_and\_delete scenario
* [Core] Add missed @six.add\_metaclass(abc.ABCMeta)
* [Verify] Updating Tempest config file
* Fix the action name about multiple murano.create\_service
* [Reports] Expand menu if there is only one menu group
* Clarify docs around results report
* [Manila] Add share-network autocreation context support
* Add Senlin plugin for profile and cluster
* [Verify] Discover tests for system-wide Tempest installation
* [Reports] Embed Rally version into HTML reports
* [Plugins] Fix vm.utils.\_run\_command\_over\_ssh
* Fix broken gate
* nested\_level should be a number
* [Verify] New command to uninstall Tempest plugins
* [magnum] Fix client
* Fix typo
* Add monasca benchmark in plugin - Part 0: metrics
* [Verify] New command to list Tempest plugins
* Fix deprecated comment
* Fix designate job
* Fix to solve None user id in existing\_users context
* modify the home-page info with the developer documentation
* Fix bash error in raally-gate script
* [existing users] Restore original quotas
* [docker] fix ability to use \`rally verify install\`
* launch tests in parallel
* Fix python3.5 unit test failure
* Make our docker-image more user-friendly
* Add performance degradation SLA plugin
* [Verify] Adding '--skip-list' arg to \`rally verify start\` cmd
* Refactor osclients
* Add default role name to rally config
* Fix simple typo mistake
* Disable 'rally-manage db downgrade'
* Add cover-master to gitignore list
* Sync requirements

0.5.0
-----

* Add the unit-test of add\_and\_remove\_user\_role
* add release notes for 0.5.0
* Modify rally requirements while preparing to release
* [requirements] Add helper for managing requirements
* [Verify] Improvements in \`rally verify installplugin\` cmd
* Fix output of small floats in results
* Add support for watcher osclient and watcher related types
* Use flavor name for checking flavor existence
* [Core] Make iterations numeration starting from \`1'
* Added auto-configure for node group
* replace assertNotEqual(\*,None) to assertIsNotNone
* Replace deprecated LOG.warn with warning
* Validate scenario arguments
* [Verify] New command to install Tempest plugins
* Add id\_attr to use in wait\_for\_status method
* Add Python 3.5 classifier and venv
* Fix a typo in Rally UI File
* [Spec] Spec class-based-scenarios.rst is implemented
* Remove unused LOG to keep code clean
* Fix test name to help to reveal its intend
* Use a shortcut to configure function
* [Reports] Conditionally reformat atomic charts layout
* Add volume type argument in volume context
* [spec] Add performance degradation SLA plugin
* Remove white space between print and ()
* Add param volume\_type when creating volume in nova scenarios
* Add \_\_ne\_\_ built-in function
* Add dns\_nameservers option to network context
* Updated const.py file with new versions of plugins
* [feature request] Check queue perfdata
* Adjust docs reference to "KeystoneBasic.authenticate" scenario
* Add keystone validate token using ceilometer scenario
* Fix bug on image\_valid\_on\_flavor
* [Plugins] Add cleanup context for security groups scenarios
* Add NoverServers.boot\_server\_from\_volume\_snapshot
* [CI] Fix coverage job
* [Scenario] Small fix in docstring
* Fix 'existing\_users' context against keystone v3
* Use pytest instead of testr
* [Plugins] Class-based scenario DummyFailure
* [spec] Making DB scalable and flexible enough
* [ci] rename rally-gate.py to wip-rally-gate.py
* Stopping server is redundant before cold-migrating server
* Make dict.keys() PY3 compatible
* Ensure consistent naming of sample scenarios
* [CI]Disable failing tests for Neutron with existing users
* Modify nested\_level from dict to integer
* Mark several cleanup resources as tenant\_resource
* [Plugins] Small but important fix in docstrings
* [Plugins] Improve scenario output from boot\_runcommand\_delete
* Add the nics argument in servers context
* Fix scenario examples for vm
* [CI] Move create\_and\_delete\_vips to unstable job
* Add boot\_and\_delete\_server samples and functional test
* Remove redundant lines
* [Core] Introduce class-based scenarios
* Modify file-exporter plugin and base TaskExporter
* [CI]Prepare for gate job testing existing users context
* Revert "Gate: prepopulate cluster with one server"
* Use Nova only when Nova is available
* [Verify] Add tempest configs and move to new section
* Updated from global requirements
* Add UUID support to \_id\_from\_name
* Create deployment conf depend on identity version
* Remove costilius module
* Update create\_nested\_snapshots\_and\_attach\_volume
* Fix required\_api\_versions validator
* Mark as deprecated distance method
* [Requirements] Remove \`fixtures' package
* [Tests] Fix: isolate some tests from values of env variables
* Fix ceilometer bug with required contexts
* Add a required\_api\_version validator
* Introduce Plugins Base classes
* Updated from global requirements
* Expose ssh timeout and interval in vm scenarios
* Make custom\_image context glance v2 compatible
* [Verify][CI] Add 'ctx-create-resources' arg to rally\_verify.py
* Make sahara\_image context glance v2 compatible
* [Install] Improve installation script
* Add cacert\_key to \_get\_auth\_info
* Add CinderVolumes.create\_volume\_from\_snapshot
* verify: start and import\_results always print uuid
* Updated from global requirements
* Fixed syntax in share\_networks example comment
* Add NovaServers.boot\_and\_update\_server
* [Spec] Rally Task Validation refactoring
* Updated from global requirements
* Updated from global requirements
* Change list\_samples to invoke new\_sample.list
* Fix CinderVolumes.create\_volume\_and\_clone
* Add name to test plugin in unit tests
* [Sahara] Missed 'endpoint\_type' parameter when creating Sahara client
* [magnum] Change  magnum service type
* [Verify] Fixing sahara plugin tempest tests
* [Plugins] Fix custom output from boot\_runcommand\_delete()
* [Spec] Small fixes in introduce-class-based-scenario-implementation.rst
* Mock generate\_cidr in unit test
* [Plugins] Add deprecation mark to dummy scenario
* Use correct parameter name for CA file with heat client
* Fix "AttributeError" when using "existing\_users" context
* [Verify] Correctly configure auth URLs and auth version
* Updated from global requirements
* Fix creating deployment fron env
* [Reports][CLI] Introduce Trends report
* [Reports] Add Jinja2 template for upcoming Trends report
* [Reports] Introduce class processing.plot.Trends
* Do not ignore cacert setting
* Add missing sla section to the input task
* Add the argument image into create\_and\_upload\_volume\_to\_image
* Remove copy paste from file-exporter plugin
* [CI] Add missing SLA to actually bugs check bugs in gates
* Gate: prepopulate cluster with one server
* Possibility to read OS\_ENDPOINT\_TYPE (OS\_INTERFACE) variable
* Add heat dataplane context
* Add Murano environments context
* Add dnf package manager support in Rally install script
* Fix typo in rally/cli/cliutils.py
* Wait for should consider pending\_delete
* [Verify] Changes related to \`rally verify discover\`
* Updated from global requirements
* [Docs] Add glossary page
* Updated from global requirements
* Fix runner timeout
* [Verify] Configure 'aodh' service in 'service\_available' section
* [gates][requirements] move gnocchiclient to optional
* Add flavor\_name variable to the nova job template
* Fix the repository with openstack
* Add possibility to balance usage of users
* Fix mistake parameter auto\_assign\_nics in nova.jaml
* [Scenario] Extend docstring for Scenario.add\_output
* Delete depracated types stuff from Rally
* [Magnum] Add Magnum client to Rally
* Updated from global requirements
* Add create-flavor and flavor-access-list scenario
* Refactor unit test for nova scenario utils
* Add "required" to RPSScenarioRunner plugin
* [Docs][Verify] Adding page for \`rally verify\` [part 1]
* Remove use of is\_temporary() in resource cleanup
* Fix random fail on rally-ci cli tests
* Use optional atomic action in proper place
* Support single-AZ deployment
* Add CinderVolumes.create\_volume\_and\_clone
* Extract redundant time field from pool in runner
* [Doc] Fix small typo in cli\_reference
* Rectify indentation in scenario YAMLs
* Fix bugs on booting a server from volume like "Change #312845"
* Add import\_opt for CONF.benchmark.glance\_image\_delete\_\*
* [Verify] Add 'version' arg to install/reinstall cmd
* install\_rally.sh: Get latest pypi links more robustly
* fix bug on booting server from volume
* [Context] Fix side effect changes of context config
* Remove redundant constrain from constant runner schema
* [Common] Add class rally.common.utils.LockedDict
* Remove unused requirements
* Fix randomly fail keystone roles tests
* Fix load duration calulation in case of 0 iterations were run
* Remove extraneous isfile check for task file
* [Spec] Introduce class-based scenario implementation
* [Common] Move code from utils.percentile to PercentileComputation
* Fix keystone unit tests
* Fix randomly failed unit test
* Fix randomly failed unit test
* [Verify] Don't install Tempest when \`rally verify start\`
* Add configs to MapR plugin
* Store iteration number in context obj
* [Install] Fix installation script
* Fix Keystone wrapper argument order
* [Verify] Removing '--tempest-config' arg from 'reinstall' command
* disable urllib3 warnings only if the library provide them
* Add the comment about cyclic reference
* Add Cinder volume\_types context
* Use glance wrapper where appropriate
* Adding gnocchi client and keystoneauth to rally
* [Scenario] Remove unnecessary cleanup
* [Docs] use openstack theme outside rtd

0.4.0
-----

* fix up the formatting of the readme for the announce script
* Fix empty config error message
* [DB][TaskResults] Save durations stats in task results
* Release notes for 0.4.0
* [Verify] Fixing 'xfail' mechanism and whole test case failure issue
* De-duplicate consume() for delete in users context
* Transform DB layer to return dicts, not SQLAlchemy models
* Use Broker Pattern for Keystone roles context
* [Verify] Fixing issue with discovering images
* [CI] Move cinder scenarios to cinder job from neutron\_unstable
* Updated from global requirements
* Friendlier error message with empty task file
* Isolate scenario args between iterations
* Use "visibility" param in create\_image() for V1
* Fix filters in glance v1 wrapper
* Refactor neutron utils
* Fix various bugs in tests/code that creates
* Fixed typo in context plugin doc
* Fix typo in install\_rally.sh script
* Use python-openstackclient as sample CLI in output of deployment create
* [Docstrings] Improvements for docstings
* Use real creadentials object at sahara tests
* Suppress warning insecure URL messages
* Add basic support for Senlin service
* Fix rally task verification log
* Print proper status in task results
* [Tests] Fix unexpected tox output of SomeError exception
* [Verify] Fixing Tempest resources context to create network resources
* Updated from global requirements
* Add functional tests to task detailed with errors
* Test: Compare resources even if SLA fails
* Add more bindings for bounce server scenario
* Convert task execution time to float value
* Updated from global requirements
* [Reports] Fix parallel iteration calc in LoadChart
* [Reports] Improve results calculation in charts.Table
* [Tests] Remove costilius from test\_charts
* [CLI] Add status messages of db migration process
* [CLI] Display task errors in human-friendly form
* Fix Zaqar client issue
* Test: Distinguish expected added resources
* Test: don't stringify cloud resources until printing
* Test: switch osresources to Credential class
* [Bug1535948] Add Heat template validator
* Glance wrapper: De-obfuscate image fetching
* [Core] Fix jsonschema huge impact on task performance
* Updated from global requirements
* Make glance web uploads streamable
* Make sure we don't have a stale package cache
* Fix import in migration test
* [ci] Workaround for siege workload template
* Remove hardcode from heat service
* Fix rally-neutron-unstable-job
* Move unstable scenario to neutron unstable job
* Remove usage of costilius in Charts
* Use existing\_users in TestTaskSamples
* [Reports] Hotfix for rendering tables in HTML report
* Restrict deletion of "default" security group
* Cleanup: Fix getting name from neutron resources
* Add exact error message into runcommand\_heat scenario
* [Reports] Fix calc of running iterations in parallel
* Show Missing Argument Error raised by Argparse
* Allow max\_avg\_sla per atomic actions
* Calculate duration of load more accurate
* Reduce default speed of users creation in users context
* Remove deprecated scenario output mechanism via returing value
* Actually delete result\_batch arrays
* [CI] add --debug flag to deployment check
* [Reports] Various fixes for load profile chart
* [Reports] Move angular directive to separate file
* [Reports] Refactor processing.plot.plot
* Fix typos in Rally files
* Support OS\_PROJECT\_NAME as well as OS\_TENANT\_NAME in the CLI
* [spec] Rally Verification refactoring
* Change deployment model schema
* Add task exporter to the file system
* [Tests] Some fixes in SaharaInputDataSourcesTestCase
* Add heat scenarios: output-show, output-list
* Fix typos in comments and docstrings
* [CI] Download sahara image from local mirror

0.3.3
-----

* Release notes for 0.3.3
* Update release notes for 0.3.2
* [Verify] Set time precision to 3 digits (instead of 5) after dot
* Print set's deprecation message only once
* devstack: support to be enabled with different plugin name
* devstack: removed uncalled code
* Add ability to select keystone API version
* [Verify] Don't use "--parallel" flag when concurrency == 1
* Cleanup threads config option
* [Spec] Improve atomic actions format
* [Plugins] New scenario Dummy.dummy\_random\_action
* Add configuration for Heat caching during validation
* Automatically configure mock context clients
* Fix validate\_args function in cliutils
* Convert existing scenarios to new types plugins
* Updated from global requirements
* Convert ResourceType classes to plugins
* [CI] Check nova is available in deployment
* Add inexact matching to name\_matches\_object
* Delete only images created by images context
* Fix glance cleanup filter
* [CI] Fix mos jobs to work with keysone v2.0 and v3
* Destroy deployment in spite of exception
* Always assign a name to created images
* Use proper attribute to get heat stack name

0.3.2
-----

* Release notes for 0.3.2
* Clean cleanup context
* Add backwards compat for server\_kwargs name change
* [docs] Add code block types into docstrings
* Create new types.convert()
* [Spec] Deployment unification
* Updated from global requirements
* [ci] Remove sla from heat-runcmd job
* Add missing \`insecure\` option to sahara client
* unify \*\_kwargs name in scenarios
* Add task\_id argument to name\_matches\_object
* Add name() to ResourceManager
* [Verify] Improve comparison of Neutron client versions
* Remove deprecated 'script' and 'interpreter' in favor of 'command'
* [CLI] Rework commands.task.TaskCommands.detailed
* [API] Extend api.Task.get\_detailed
* [Cliutils] Skip "n/a" value in cliutils.pretty\_float\_formatter
* [Verify] Fixing Tempest config generation
* [Docs] Rework "Rally Plugins Reference"
* [CI] Fix rally-mos job to work with mos-8.0
* Update the scenario development section
* Updated from global requirements
* Change unstable neutron rally job's flavors
* Update runner and context sections of concept
* Refactored atomic action in authenticate scenario
* [Cliutils] Add dict support for cliutils.print\_list
* [Tests] Improve test\_cliutils.py
* Fixed imcompatible argument in service create with keystone v2/3 API
* Documentation tox fix
* Add support for nova service-list
* Change structure of rally plugin and plugin references page
* [Docs] Add cli reference to docs
* Designate V2 - Add recordset scenarios
* Avoid using \`len(x)\` to check if x is empty
* Updated from global requirements
* [Verify] Adding 'deprecate' warning to \`rally verify start\`
* Make Rally cope with unversioned keystone URL
* Expand Nova API benchmark in Rally
* [Bug1509027] Add failure statuses into HEAT utils
* [Docs] create utils for sphinx extensions
* Updated from global requirements
* [Verify] Removing deprecated arguments
* [Doc] Make documentation for output plugins
* Make sure VolumeGenerator uses the api version info  while cleanup
* Update URL in templates
* [spec] Refactoring scenario utils
* Overhaul SeekAndDestroy.\_get\_cached\_client()
* Using 'dt' as alias for datetime imports
* Quoting openrc values
* [Verify] Deprecate for removal '--tempest-config' for 'reinstall' cmd
* Updated from global requirements
* Add ironic to services list
* Use new-style Python classes

0.3.1
-----

* Release notes for 0.3.1
* Revert "Make Rally cope with unversioned keystone URL"
* Fix cleanup manager with api\_versions context
* Restore ability to pass kwargs to glance \_create\_image
* Fix install\_rally.sh to get it to work on MacOSX
* [Verify] Don't create new flavor when flavor already exists
* Test: Stop using non-existent method of Mock
* [Sahara] Fix flavor for cloudera manager
* Fix config file generation after Glance wrapper addition
* Pass api\_versions info to glance images context
* Fix release notes for v0.3.0

0.3.0
-----

* Add v0.3.0 release notes
* Add wrapper for Glance API versions
* Avoid using range() of len() to get index of iterable
* Ensure cleanup context uses the supplied api\_versions in the task
* [Verify] Don't create new image when image already exists
* Make install\_rally.sh --branch accept git tree-ish
* Fix quotas to use the supplied context
* Fix volumes to use the supplied context
* Remove unnecessary second argument from get method
* Add heat siege workload scenario
* Misc. install\_rally.sh improvements
* Log VM console when Rally fails to connect to VM
* Updated from global requirements
* Updating the rally.conf.sample file
* [Verify] Adding new command \`rally verify discover\`
* [Verify] Configure the 'auth\_version' option
* Add base class for task exporter
* Fix liberty Neutron client incompatiblity
* Fix tempest config creation on Kilo
* Use booleans to check for empty lists
* Add the scenario of list-flavors
* Fix Ceilometer scenarios against Keystone API v3
* Move OpenStack cleanup utils out of context
* Re-run failed Tempest tests
* Make Rally cope with unversioned keystone URL
* [CI]Split Rally gate scenarios for Keystone API v2.0 and v3
* Removed mutable default argument
* Fix unit tests for authenticate scenarios
* Updated from global requirements
* [CI]Restore disabled scenarios in Rally gate job
* Add Cinder v1 test to gate
* Fix typo
* Fix gates against Keystone V3
* Ensure image['..'] does not throw KeyError
* Integrate Rally & Alembic
* Remove fuel deployment engine
* Add note about the fact that Rally won't use testtools' assertions
* Remove the extra 'g' in description for plugin show
* Default cinder to v2
* [Reports] Custom labels for "Scenario Output" charts
* Updated from global requirements
* Update plugin image in documentation
* Fix RST formatting in a spec
* Remove unused import and LOG variable from rally.exceptions
* Switch pip version pinning from <8 to !=8
* Move "tempest concurrency" feature request to implemented
* Remove \`None\` as a redundant second argument to \`dict.get()\`
* Updated from global requirements
* [Ci] Do not buffer subunit output for job gate-rally-dsvm-cli
* Updated from global requirements
* [Verify] Fixing 'test\_server\_basicops'
* [cli] Remove "rally info" command
* Force usage of pip < 8 version
* Fix deprecated import of rally.common.log
* Deprecated tox -downloadcache option removed
* Consistent resource names spec is implemented
* Improve rally cli docstrings
* Remove duplicate key in dictionary
* [verification] separate Horizon availability check
* Regenerated the sample configuration file
* Do not ignore unknown service\_types
* [docs] remove reindentation from plugin descriptions
* Pass environment variables of proxy to tox
* [Sahara] Improved flavor paramter for clusters
* [Gates] Avoid duplication in logging
* [docs] Increase an understanding of api\_versions context
* [Gates] Remove redundant args from \`rally verify ...\` commands
* Fix release 0.2.0 notes

0.2.0
-----

* [Spec] Refactoring Rally Cleanup, Part II
* [gates] Speed up gate-rally-dsvm-verify-light
* [Scenario][Reports] Improve scenario output format
* Add release notes for 0.2.0
* [Verify] Deprecating --no-tempest-venv for \`rally verify [re]install\`
* [Verify] Use 'generate\_random\_name()' method for resource names
* [verify] fix rally task for verification
* Remove EC2 from Nova gate job
* [Spec] Update and move spec about new output format
* Additional argument support for CinderVolumes
* [Reports] Remove class OutputStackedAreaDeprecatedChart
* [Reports] Improve HTML report and add dummy\_output scenario
* Let TempestResourcesContext implements utils.RandomNameGeneratorMixin
* [Reports] Make HistogramChart.render() suitable for JS
* [Scenario] Add method \`Scenario.add\_output'
* Use assertTrue/False instead of assertEqual(T/F)
* Remove openstack-common.conf
* Updated from global requirements
* Fix hacking checks
* Images context: accept arbitrary image create args
* Test image downloading in image\_context
* [VM] refactor wait\_for\_ping
* Split hacking check for db and objects call from cli
* Fix for the deprecated library function
* Rally: get the correct IMAGE\_NAME from cirros\_img\_url
* [Docs] Remove \_build directory before main cmd
* [Docs] Fix metavars
* [Docs] Point a release of deprecation for info cmd
* test: make enforce\_type=True in CONF.set\_override
* [cli][ux] Make rally task report "--out" optional
* Updated from global requirements
* Put py34 first in the env order of tox
* [Docs] improve cli docs
* Deprecate show CLI commands
* [Scenario] New output format schema
* [Reports] Add charts classes for new output format
* Fix some inconsistency with docstrings in rally
* Security group for OpenStack server provider
* [Reports] Rename benchmark\_info to workload\_info
* Python 3 deprecated the logger.warn method in favor of warning
* [Verify] Configure 'catalog\_type' in 'data-processing' section
* Wrong usage of "an"
* py33 is no longer supported by Infra's CI
* Use interruptable sleep in sleep\_between method
* Drop py26 from tox
* Remove extra link in \`All release notes\`
* [Verify] Moving log message under context
* [Verify] Deprecating --system-wide-install in favor of --system-wide

0.1.2
-----

* Relese notes 0.1.2
* [Verify] Adding "xfail" mechanism for Tempest tests
* [verify] Fix setting testr arguments
* Updated from global requirements
* install-rally.sh uses "mysql+pymysql://"
* [Reports] Add abstract class Table class and improve MainStatsTable
* tests: add ContextTestCase that mocks clients
* [StreamingAlgorithms] result() returns None if data is missing
* Add hacking rules for CLI import
* Adds Designate domain-update test
* install\_rally.sh: Add redhat-rpm-config to package list for EL distros
* Remove rally show commands from rally gate job
* Fix placements of docs, samples, certification in the Dockerfile
* [Tests] Fix test\_get\_creds\_from\_env\_vars\_when\_required\_vars\_missing
* keep symbol consistent in code
* Do not remove SOURCEDIR in install\_rally.sh
* Fixed Glance Certification part
* Make Neutron router scenarios work with network context
* Make PluginWithSuchNameExists more informative
* specify required services for all resources
* [Verify] Rework help messages for \`rally verify\` and small improvements
* Fix Rally Version in Docs
* Adding Cue client to osclients
* [Verify] Adding possibility to specify concurrency for tests
* Moving "os.environ" things to envutils.py module
* [Verify] Don't install Tempest when \`rally verify [gen/show]config\`
* change name of rally.common.log module
* [feature request] Add possibility to specify concurrency for Tempest
* Possibility to read OS\_INSECURE and OS\_CACERT env variables
* Prepare for distribured runner, part 1
* Designate V2 support - part 2
* [Sahara] Update of EDP samples
* Remove excessive quotes in \_revert\_resize()
* Fix format of existing network in \_get\_or\_create\_network()
* [Sahara] Added spark and ambari plugins
* [Neutron] Fix \_get\_or\_create\_subnets() return value
* [StreamingAlgorithms] Unify exception type raised by result()
* [Sahara] Fix for Sahara cluster context
* [gates] move ceilometer alarts scenarios to non-voting job
* [Sahara] Remove volumes from non-workers
* Make disk\_format configurable
* Make param description & LogInfo better
* Add Workload object
* Minor change in docstring
* Rename "endpoint" to "credential" (part 4)
* [Verify] \`rally verify install/reinstall\` with --no-tempest-venv
* Add batching for samples creation in context
* Add 'merge' functionality to SLA
* Rename "endpoint" to "credential" (part 3)
* [CI] Remove HttpRequests scenario from MOS job
* Fixing incorrect work of the \_initialize\_testr method
* [requirements] Change requirements to support py26
* Remove db and object usage in cli.commands.deployment module
* [Verify] Fixing failures of test\_server\_basicops
* [Verify] Fixing issue with create\_isolated\_networks = False
* [Verify] Using subunit-trace instead of colorizer.py
* Add Designate API V2 support - part 1
* Remove db and object usage in cli.commands.verify module
* [Verify] Making all names of columns consistent in HTML report
* [Verify] Fixing issue with failures of Heat tests
* [Verify] Fix total time
* Fix timeout for scenario runners
* [Verify] Set run\_validation to True to enable the corresponding tests
* Extend statistic benchmarks for Ceilometer
* [api\_versions] Allow to use unknown services
* [Verify] Reworking unit tests for config.py [part 2]
* Rename "endpoint" to "credential" (part 2)
* [Verify] Fixing typo in the name of a variable
* [Verify] Make total time like H:M:S (<time in sec> s) in HTML report
* Updated from global requirements
* Replacing application\_catalog with application-catalog
* Remove db and object usage in cli.commands.task module
* Add region\_name when generate tempest conf
* Add region\_name in devstack plugin script
* [verify] Remove redundant script
* Remove unnecessary shell=True in subprocess calls
* [verify] Fix calculation number of tests
* Remove old random name generator
* [verify] add ability to turn off compare in job
* [Verify] Reworking unit tests for config.py [part 1]
* Remove old generate\_random\_name() use from sahara context
* [CI] Remove hardcoded hadoop version from mos task
* Add orchestration section to tempest.conf
* Using 'ready\_statuses' instead of 'is\_ready'
* [CI] Join together mos and mos\_neutron files
* [Reports] Add an option for embedding JS/CSS libs into report
* Rename "endpoint" to "credential" (part 1)
* Implement new random name generator for wrapper plugins
* [Ci] Check Heat service availability in tests.ci.osresources
* [Verify] Fixing issue with SSL deployments
* [report] Fix the dummy mismatch in report.mako
* Updating rally.conf.sample
* Fixing spelling errors in doc strings
* Updated from global requirements
* Fix floating ip setting name in samples
* Fix some typos by toolkit typo
* Extend samples list benchmarks for Ceilometer
* [Spec]Add a spec for distiributed load generation
* Drop python 2.6 classifier
* Updated from global requirements
* Fix broken logic in some neutron scenarios
* Stop discarding exit codes
* [Sahara] Move sahara data to sahara context
* [Spec] Propose improvements for scenario output format
* [Ci] Get changes in resources after task run
* [Sahara] Add Swift to output data sources
* Updated from global requirements
* [Verify] Refactoring HtmlOutput
* OpenStack typo
* [Verify] Fixing tokens expiration issue when deleting Tempest resources
* [Verify] Adding creation of network resources to Tempest context
* Change install script to use git.o.o
* Extend list benchmarks for ceilometer
* NotFoundException not showing exception message
* install Rally in existing virtualenv
* Rename BenchmarkEngine to TaskEngine
* [Verify] Don't mask original exc if failure happens
* Improve rally.common.utils.RAMInt code
* Remove Extra dashes in "missing argument" error
* Implement subunit\_parser
* Updated from global requirements
* [verify] move report template for job to jinja2
* [Swift] Add additional Swift scenarios
* Fix docker build command
* Use network context for Neutron scenarios
* [Sahara] Fix sahara gate
* [verify] Add new script to launch verification job
* Add quotation marks
* [Reports] Colorize Pie chart errors in red
* remove default=None for config options
* Fix test\_get\_template\_\* tests
* [spec] Improve and make pluggable type conversion
* Display correct version
* [Spec] Task and verification export
* [Reports] Switch task HTML report to Jinja2 templates
* Add timestamps and metadata into creating samples
* Validate arguments for command rally verify compare
* Add boot-server-from-volume-and-resize for nova benchmark
* Add wrapper to support different API versions
* Fix 'merge lists'
* [Sahara] Add Swift to input data sources
* [Sahara] Split Data Sources context
* Fix flavor name on Rally documentation
* Make name\_matches\_object accept list of objects
* Add AST checker for missing @ddt.ddt decorators
* Support jinja2 in rally.ui.utils
* Adds nova show and nova console output plugins
* Creation a list of samples in sample generator
* Add 'merge lists' function
* Add boot-server-attach-created-volume-and-resize nova benchmark
* Refactoring log utils
* [osclients] Add timeout to keystone session
* [Sahara] Added latest versions of sahara plugins
* Add missing ddt class decorator
* Updated from global requirements
* [Verify] Moving compare.mako to rally/ui/templates/verification
* Fix \`raise\` section in docstrings
* Fix sahra-dsvm and sahara-mos jobs
* [Reports] Fix tracebacks from rally.ui.utils run via CLI
* [Sahara] Add ability to specify service\_type
* Add Neutron Security Groups scenarios
* Fix random name bug if task uuid is too short
* Check nova v2.2 in gates
* Updated from global requirements
* Added workload formulas
* Enable ddt on random name generator tests
* Add new context plugin to support different API versions
* Add ability to specify service types for clients
* Add support for Designate V2 api
* Move rally.osclients.Clients to plugin base
* Implement new random name generator for remaining scenario plugins
* Use local.conf instead of localrc
* [Reports] Reduce histogram charts weight to make report faster
* Fix Murano delete environment
* Refactoring task detailed command
* Add backwards compat for Keystone name\_length
* Updated from global requirements
* Add validate monasca service to authenticate.py
* Prevent false negative results during wait for resource deletion
* Fix Sahara service type
* Add neutron jobs to certification task arguments
* Add parameter --tests-file to 'verify start'
* Implement new random name generator for keystone scenarios
* Convert scenarios to optional\_action\_timer
* Add CobblerProvider provider
* [Sahara] Split EDP context
* Updated from global requirements
* Changing URL to repo to pull Tempest from
* Adding command to show tempest.conf
* Capture timing information for failed atomic actions
* [PLUGINS] Follow symlinks in plugin discovery
* Use sed without -i option for portability
* Fix random miss in comsumer/publisher broker
* Certification task: neutron checks
* Fix the order of iterations in DB
* Fix sshutils for error in writing received data
* New optional\_action\_timer decorator
* Make Docker Container work out of the box
* Display less OS specific message in install script
* Add task format converter
* Fix nested atomic action in VMTask scenario
* Use wait\_for\_status instead of wait\_for\_delete
* Scenario list\_images doesn't accept args
* Implement new random name generator for context plugins
* Change message of log\_deprecated decorator
* Extract random nested level from Cinder scenario
* [reports] handle console.html.gz
* Add Neutron LBaaS V1 healthmonitor scenarios
* Mark wait\_for\* functions deprecated
* [cleanup] remove vip before removing port
* Updated from global requirements
* Fix cleanup of nova networks
* Refactored task.yaml
* Configure Tempest to use isolated tenants
* Fix traceback when generating output with no iterations
* [Fuel] scenario for add and remove node
* Generate report at the end of rally-dsvm job
* [Swift] Add objects context class
* Better random resource name generator
* Updated from global requirements

0.1.1
-----

* Release notes v0.1.1
* atomic: public names generator
* [Sahara] Add support for proxy node
* verification: set timeout for requests.get when testing for Horizon
* Updated scenario boot-and-delete-server-with-keypairs
* Fix & Restore cover job
* config: fix missing benchmark opts
* Reworked script generating tempest.conf
* workload: fix and rename benchmark script
* workload: fix custom\_image cleanup
* Replacing 'print' with 'LOG.xxx'
* Updating Tempest test sets
* Fix install\_rally.sh set 777 access to sqlite file
* Add resource fault detail in exception
* Small improvements for \`rally verify import\`
* Nova-network: disassociate networks before deletion
* Fix Rally reports table stats calculation
* Fix LB create and delete VIPs failure when concurrency > 1
* Fix overly narrow isfile check on tasks
* Add task argument to network wrapper
* Remove psycopg2 from requirements
* Run test\_task\_samples in separate sqlite db
* Remove usage of Exception.message
* Imporve error message if Rally can't connect to DB
* Replacing 'import\_file' with 'import\_results'
* Certification task: Add readme file
* Adds nova image list plugin
* Make code blocks in documentation prettier
* Remove task validation from api.Task.start

0.1.0
-----

* Add Neutron Floating IP benchmark scenarios
* Import tempest results into Rally database
* Document --plugin-paths/RALLY\_PLUGIN\_PATHS
* Fix project info page at the docs
* Release notes v0.1.0
* [Murano-Apps] Fix Rally deployment by Murano
* Support to include other template in task file
* Add 'Who Is Using Rally' paragraph to the docs
* Add new Murano scenarios
* Set smoke in rally-jobs/rally-neutron.yaml
* [Reports] Fix math domain error with 0 load duration
* Disable smoke-test mode for Neutron gate job
* Use Rally api in functional tests of task samples
* Certification task: load tests (macros and keystone)
* Moving commands for (re,un)instaling Tempest to "verify" category
* Implementation of 'rally task abort' command
* Certification task: reference task arguments file
* Certification task: glance scenarios
* Remove quotes from subshell call in bash script
* Certification task: quota scenarios for cinder, nova
* Certification task: nova checks
* Certification task: keystone checks
* Certification task: macros and cinder checks
* [docs] Imporve Plugin Reference
* Add context to plugin tests
* Remove required admin=True in LB VIP scenario
* Add missing lbaas extensions check
* Updated from global requirements
* [Reports] Fix the namings of things in Rally report
* [cli] Remove "rally info" command
* Tempest: use mkdtemp to make temp directories
* Make network context more configurable
* [COVERAGE] Turn off coverage
* Add test perttern for cliutils.make\_table\_header
* Adding possibility to generate tempest.conf, not running tempest
* Fix lbaas vip cleanup with LbaasV1Mixin
* Make default do\_use=True in start method
* Auto assign nic for servers context
* [Reports] Add Load Profile chart to report
* Fix typo in the README.rst in tests/ci/
* [docs] Add plugin reference
* Fix users/tenants in Neutron job and sample tasks
* Add usage of 'os.path.expanduser' to file\_exists validator
* [cli] Introduce "rally plugin" show and list commands
* Updated from global requirements
* [VMTask] Avoid Floating IP leakage
* New nova jobs: boot\_and\_delete\_multiple\_servers
* Revert "[JOBS] disable several ceilometer scenarios to unblock gates"
* [Reports] Improvements and refactoring in HTML report UI
* Allow non virtual environment option for verify
* [Reports] Use new reports charts classes
* Add create\_and\_update\_secgroups scenario
* [feature request] Check queue perfdata
* Updated from global requirements
* Fix sqlite file permissons in installation script
* [JOBS] disable several ceilometer scenarios to unblock gates
* [Cleanup] FloatingIP
* Fix typo in rally/consts.py
* Add Neutron Loadbalancer v1 create and update vips
* Add Neutron LoadBalancer v1 create and delete vips
* Add Neutron LoadBalancer v1 create and list vips
* [Reports] Add method objects.Task.extend\_results()
* Changed wrong abbreviation used for PTL
* Nova: Support auto\_assign\_nic when booting multiple servers
* [BugFix] Add ability to install tempest in docker
* Add check for neutron extension support in utils
* Updated from global requirements
* Change install docs to fix wrong command
* [JOBS] Return dummy scenario for nova quotas
* Add usage of os.path.expanduser to Murano context
* [Bug-Fix] use project\_id while creating networks via nova-net
* [Fuel] Environments scenarios
* Fix the rally gates doc link
* Fix the installation steps for rally
* Do not remove network from dhcp agent if they are not supported
* [gates] Convert Rally to the new Devstack plugin model
* [spec] Consistent resource names
* Fix typos in Rally code
* Fixes variables in Tempest generated config file
* Add attachment-related scenarios to test on gates
* [Bug-Fix]: Remove ability to put incorrect kwargs for an exception
* Fix argument name in Murano context
* Add specific list method for NovaServer res
* Fix the typos and description of functions
* Remove notes about aliases for 'rally use'
* Updated from global requirements
* Fix lbaas resources cleanup
* Fixed description sentences of functions
* Fix typo in the code comment
* Fixed some typos in the files
* Fix incorrect code comment in api.py
* Removing unused dependency: discover
* [feature request] Add abbility to caputre logs from tested services
* [feature request] Allows specific repos & packages in install-rally.sh
* Fix one typo on Rally tutorial document
* Updated from global requirements
* Fix the scenario plugin sample
* Updated from global requirements
* Fix typo in Rally Installation document
* Move & refactor atomic actions
* Switch scenarios plugins to plugin base
* Set samples default flavor to m1.tiny
* Add Cinder create\_and\_update\_volume scenario
* Fix typo "resources"/"resourses" in rally/cli/commands/show.py
* Updated from global requirements
* Word spells have been corrected
* Spell mistakes have been corrected
* Word spells have been corrected
* command is elaborated cd --> change directory
* Word spelling have been corrected conficuration --> configuration
* Create pool in lbaas context with correct tenant\_id
* Quick fix for broken dependency on bare-centos VM
* Fix some CI task files
* Add FilePathOrUrlType
* Add release notes section in Rally's doc
* Expose config for timeout of resource deletion
* Updated from global requirements
* Don't overwrite existhing venv dir when using --yes
* [spec] Refactoring Rally cleanup
* Add boot\_runcommand\_delete\_custom\_image scenario
* Move existing\_network context class to proper directory
* Nova: make nova server scenarios easier with existing users
* Rename Manila context
* Move auto\_assign\_nic to class method
* add .eggs to git ignore file
* OpenStackProvider: Optionally wait for cloud-init
* Add existing\_network context
* Better error messages from invalid sample tasks
* Updated from global requirements
* OpenStackProvider: Refresh IPs before looking for access IP
* OpenStackProvider: Purge disabled code that reads arbitrary files
* Refactoring of scenario plugin base
* Expand script path by os.path.expanduser
* Fix six typos and grammar inaccuracies in Rally Readme Files
* Fixed create\_user\_update\_password scenario with keystone v3 endpoint
* Updated from global requirements
* [Reports] Small simplification in new HTML charts classes
* Fixed insecure and cacert not passed to user context
* RPS thread start pause correction
* Fix sample json file
* Add EC2 api scenario that list all servers
* Use os.path.expanduser in file\* types
* [Manila] Add share networks context for existing users
* Add kwargs to list\_pools function
* Make context plugins decide how to map their data for scenario
* [Reports] Add classes for task HTML report charts
* Refactor BenchmarkEngine
* Updated from global requirements
* Update Dockerfile to use new install\_rally.sh
* Add context that creates servers using EC2 api
* Place rally.conf under virtualenv
* Use config file for genconfig and update sample config
* Add a validation check for NovaFloatingIpsBulk scenarios
* This submission fixes typos for the following files -
* Modify atomic action timer in lb scenarios
* Update output of 'rally show flavors' in tutorial
* Update to latest available CirrOS version 0.3.4
* Fix item name in new task format spec
* Add Nova network scenarios
* Move rally/objects -> rally/common/objects
* Fix instance name created by vm scenarios
* Extend neutron port cleanup
* Fixes Typos in rally files
* Move rally/db -> rally/common/db
* Updated from global requirements
* Neutron: Make resource update methods more generic
* Rename EngineFactory -> Engine
* Add unit test to check create\_router with ext\_gw
* Rename context.context -> context.configure
* Refactoring cinder create\_volume\_backup scenario
* [Bash-Completion] Fix bash completon for files paths
* Change endpoint initialization
* Add image\_command\_customizer context
* Updated from global requirements
* [gate] workaround to run tempest's setup.py
* Add Cinder create\_and\_list\_volume\_backups scenario
* Add ability to specify directory with murano package files
* Remove call to nonexistent NoSuchContext exception
* [Docs] Small rename s/plot2html/report/ in docs
* Fix eight typos on Rally documentation
* Adds LBaaS context, verify network extension
* Add Ironic scenarios
* VMTasks: Make waiting for ping timing configurable
* Updated from global requirements
* [Common] Add more streaming algorithms
* openstack: make scenario able to reuse clients
* OpenStackProvider: add option to specify region
* OpenStackProvider: Allow userdata using config\_drive
* OpenStackProvider: Prefer access and public IPs for VMs
* Updated from global requirements
* Use proper fixtures for manila utils tests
* Add OS\_<PROJECT, USER>\_DOMAIN\_NAME to openrc
* Reorganize Openstack context plugins
* [Manila] Add scenario for attach of security service to share network
* Updated from global requirements
* Add command-dict option to specify command args
* Add command-dict option to upload a local command
* Replace \`id\` with \`id\_\`
* Keystone: Add scenarios to benchmark user disable/enable
* Modify unit tests to DDT format
* Keystone: do not allow specifying service names
* Introduce rally.plugins.openstack.OpenStackScenario
* Remove osclients from runner and scenario signature
* Fix typo docs at User stories
* Make \`boot\_runcommand\_delete' accept command-dict
* unit-tests: inception: add mock check unit-tests
* Change neutron command to validate token
* Updated from global requirements
* Turn on coverage job
* Change step\_8 docs to fix link to sample scenarios
* Change plugin docs to fix mislead about sla plugin
* Fix broken unit-tests
* Change step\_1 docs to fix sample command output
* Add unit test for common.utils.EnumMixin
* Improve hacking rule that checks that [] & {} are used
* [Reports] Add class for merging data stream
* Some files missing newline at end of file
* Revert "Disable sahara scenarios from dsvm-rally"
* Use die() consistently in install\_rally.sh
* Add ANSI color to heredoc output
* Add failure tag to JUnit report
* Optimize Rally imports to reduce start time
* [Manila] Add scenario for creation and deletion of security services
* Catch a connection refused exception not to show traces
* Display default values of scenario paramters
* Disable sahara scenarios from dsvm-rally
* Add Heat scenario to test scaling policies
* Add Keystone ec2-credentials scenarios
* Fix EC2 utils and tests
* Better Scenario test case fixtures
* Replace 'unicode()' function with 'six.text\_type'
* Refactor rally/deploy -> rally/deployment
* Rename rally/benchmark to rally/task
* Updated from global requirements
* Install newer pip in install\_rally.sh
* install\_rally.sh: always use wget, not cURL
* Restore coverage test functionality
* Remove extraneous failed server/image tests
* find should use portable options
* unit-tests: fix mock check errors messages
* Fix mistral client requirement
* Move CeilometerResource.get\_tenant\_resource to unstable
* Fix wrong key in msg\_fmt at DummyScenarioException
* Support validate parameters list for restricted\_parameters
* Updated from global requirements
* Do not delete Constraints in case of sqlite
* install\_rally.sh: permit absolute paths for sqlite db name
* Allow shell calls to fail around getopt checks
* Handle NoneType error from pool\_create\_args
* [Heat] create-snapshot-restore-delete stack scenario
* Optimize task table order
* Keystone get\_entities scenario backwards compat
* Add specs in doc/README.rst file
* Fix typo in ServiceType map
* unit-tests: add mock decorators usage test checker
* unit-tests: fix mocks' naming
* [Manila] Add benchmark for Manila list share servers operation
* [Heat] Add rally template with Heat Resource Group
* [Manila] Add scenario for creation and listing of Manila share networks
* [Manila] Add scenario for creation/deletion of Manila share networks
* Don't change global environment in functionaltests
* Fix timeout option in Cinder upload volume util
* Fix python ironicclient default version
* [Manila] Add scenario for creation and deletion of Manila shares
* Add Neutron Loadbalancer v1 create and update pools
* Add Neutron Loadbalancer v1 create and delete pools
* Remove rally.deploy.\_\_main\_\_ hacks
* Add rally.common.plugin.discover module
* [Floating IP] Remove port creation on fip creation
* Fix sqlite file permission for system-wide install
* [Sahara] More debug logging
* Improved wait\_for benchmark helper
* Do not update the average when iteration failed
* Remove calls to non-existent 'warn' in install\_rally.sh
* [Ceilometer] Add  scenario for events and traits
* Add smoke mode for neutron jobs
* [Manila] Add context for Manila quotas
* [Manila] Add base benchmark for list shares operation
* Add role assignment in keystone wrapper for both v2 and v3
* Add Neutron Loadbalancer v1 create and list pool
* Change "find" command path in tox.ini
* Overhaul ceilometer utils unit tests
* Remove deprecated code
* utils: make parse\_docstring respect multi-line
* Decrease time of rally verify job
* Fix formatting of maintainers section in docs
* Better mocking for self.[admin\_]clients in Scenario tests
* Install data files in proper path
* Eliminated string formatting with single mapping key
* Switch to plugin base deployengines and serverproviders
* Remove 'pyc' files before running tests
* Modify GetResourceErrorStatus exception
* Add validator to check parameter inside a dictionary argument
* Switch to plugin base: context, sla, runners
* Further improvment of plugin base
* Feature request improve install script
* Split slow rally job to several small ones
* Fix super() invocation in DBTestCase.setUp()
* Introduce command-dict and validator
* Fix Designate record deletion utility function
* Move rally.benchmark.scenarios.utils
* Move rally.benchmark.context.base module
* Updated from global requirements
* Add maintainers page in documentation
* Change new input task format specification
* Use absoulote path to venv in installation script
* tempest: add \`admin\_domain\_name' to generated conf
* Add functional test for --plugin-paths parameter
* Make plugins location configurable
* Authenticate scenario unit test improvements
* More verbose exception when ssh fails
* Removed content from rally/benchmark/\_\_init\_\_.py file
* Add Nova scenario to boot multiple servers in one request
* Move rally.benchmark.runners.base module
* [Fix] Fix issues in verification.tempest.diff
* Fix cinder scenario: create nested snapshots
* Add scenario to modify Cinder volume metadata
* [wrappers] Move wrappers under plugins.openstack
* [Heat] Pass parameters, files and env  for heat stacks
* Hide admin password in rally deployment show command
* Fix document about Rally with DevStack all-in-one installation
* Add check for users in "deployment check"
* Change logic of wait\_for\_ping
* Fix reinstalling in rally install script
* Minor typo fix and added selinux disable command
* task report: generate a JUnit report
* Allow for default() filters for jinja variables
* Add support for PyPI/PIP proxy
* [Fuel] Add Fuel scenario \`list\_environments'
* Add checks for SuSE based systems
* Rename sla.base module to sla
* Improvement of cover script
* Remove break from case statement to allow for parsing of CLI args
* Set Python2.7 as basepython for testenv
* Nova: Refactor utils.\_boot\_servers to generate its own name prefix
* Add sshutils.\_put\_file\_shell
* Add skip case for not installed clients into functional tests
* [Clients] Add test for osclients.cached
* [Feature request] Explicitly specify existing users for scenarios
* Handle http response code 404 properly
* Add keystone benchmark scenarios for roles
* Rename rally.cmd to rally.cli
* [Spec] Move plugins spec to implemented
* [Validation] Allow \`required\_clients' validate admin clients
* [Context] Users cleanup should not fail without networking
* Add Cinder create\_and\_restore\_volume\_backup scenario
* [Scenario] Split Scenarios - P6
* [Scenario] Split Scenarios - P5
* [Scenario] Split Scenarios - P4
* [Scenario] Split Scenarios - requests
* Server Group quotas support
* Minor install script improvements
* Add functional test for task samples
* [Scenario] Split Scenarios - P3
* [Scenario] Split Scenarios - P2
* [Scenario] Split scenarios - P1
* [CTX] Moves context.cleanup under plugins
* [CTX] Split context into plugins
* Fix installation on python3-only systems
* Fix wrong apt-get -y option in install script
* [Docs] Existing users and other fixes
* Added configuration parameter for cirros base url
* Fix \`sshutils.put\_file' to use sftp context
* Add E126 pep8 rule

0.0.4
-----

* Release 0.0.4
* Fix security groups cleanup
* Change default prefix in \_generate\_random\_name()
* Improve installation script
* Add Cinder create\_backup volume scenario
* Make "random" name more random
* [Docs] Update meetings info
* Add deploy environment murano benchmark
* Fixup resize\_server docstring to match reality
* Updated from global requirements
* [Sahara] Improve Image context
* Split runners to base and plugins
* Split SLA to base and plugins
* Adds tempest re-install to API/CLI
* Fixes devstack repo
* Remove executable bit from files that should not be executable
* Fix grammatical and formatting errors in documentation
* [gates] Upated cirros image version in heat templates
* [Clients] Rework osclients.Clients.register into decorator
* Fix flavor issues in samples
* Add MuranoPackage context
* Fix docstring from nova and typo in tutorial
* Setup volume detach timings separately
* Speed up and reorganize rally jobs
* Updated from global requirements
* Fix Mistral unit tests
* Updated from global requirements
* Fix RALLY\_SCENARIO filename in rally-gate.py
* Creates "plugins" tree
* Remove sla from Heat scenarios in MOS job
* Fix broken samples scenario for CinderVolumes.create-and-attach-volume
* [Spec] Reorder plugins
* Fix validation for rps parameter
* Add Nova lock and unlock scenario
* [Clients] Add classmethod osclients.Clients.register
* [Ceilometer] Add context and scenario for resources
* Cleanup heat stacks 1 time per tenant
* Set default value for updated\_at
* Updated from global requirements
* Add Nova scenario to boot and associate a floating IP
* Add Nova scenario to list hypervisors
* Remove \_delete\_all\_servers() nova utils function
* Shifting \_generate\_random\_name() call to utils
* Add test for deployment check
* Print Rally debug logs in Rally dsvm gate jobs
* Make vmtasks network\_wrapper code less wonky
* Restore coverage job
* Fix pbr and it's version checks (unblock gates)
* Fixes a typo in the tutorial
* Adding FunctionalMixin class
* Support size range in Cinder create\_volume and extend\_volume
* Add streaming algorithms and SLA to check for outliers
* Add Http Request Scenario
* Change file loading mechanism for heat tests
* Add Ceilometer list samples
* [report] Improve reports data and units
* Split validation.file\_exists, allow \`required' arg
* Add \`LogCatcher' context manager
* Extend \`sshutils' with \`put\_file'
* Fix \`sshutils' to execute commands with args
* Adds Nova floating IPs bulk tests
* Fix iteration info print of 'rally task detailed --iterations-data'
* Fix instance\_dd\_tests.sh always reading 1MiB
* Finish support of benchmarking with existing users
* Fix rally show commands for existing users in deployment
* Install optional-requirement packages for Rally docker image
* Fix coverage job to compare proper git diff and make it more strict
* [Sahara] Fix the config descriptions
* [Feature request] Specify what benchmarks to execute in task
* Fix creating user in keystone v3 wrapper
* Improve coverage job
* Add function decorator \`log\_deprecated\_args'
* Add sample for the server block migration scenario
* Add keystone update\_user\_password scenario
* Fix rally plugins relative imports
* Add keystone create\_update\_and\_delete\_tenant scenario
* Add Nova shelve and unshelve scenario
* [Docs] Various fixes
* Update stackforge to openstack
* Update git review + hosting/mirroring links
* Update .gitreview file to reflect repo rename
* Make coverage job voting
* Don't assign any roles to created users by default
* Add sleep\_between in live migration scenarios
* [Swift] Add base for Swift API Benchmarks: Patch-2
* Data structs declared with constructor replaced with literal

0.0.3
-----

* Release 0.0.3
* Add aborted and broken flags to sla checker
* Fix consuming in broker
* Add scenario to boot and rebuild server
* Add to constant and rps runneres limits for maximun Core usage
* Remove DeployEngines in favor of DeploymentEngine
* Replaced deprecated 'use deployment' with 'deployment use'
* Add create\_and\_upload\_volume\_to\_image scenario to rally-neutron job
* Replaced incorrect BenchmarkScenarios list command in doc
* Remove keystone v3 jobs from MOS
* Updated from global requirements
* Add required\_cinder\_services validator
* Fix kwargs in nova migrate scenatio
* Specify alt\_user in tempest config
* [Vm] Restore fixed IP support in boot\_runcommand\_delete
* Add heat list\_stack\_and\_events scenario
* Add keystone create\_and\_list\_services scenario
* Remove redundant option "detailed: True"
* [Docs] Remove subsection numberings in the Rally tutorial
* Add "pause and unpause" scenario for Nova
* Add unit test for keystone.\_get\_service\_by\_name
* Pass username/password to client
* Remove redundant type
* [Docs] Add tutorial chapter about task templates
* configure ssl in deployment config
* Adds endpoint to --fromenv deployment creation
* Add "suspend and resume" scenario for Nova
* Add murano benchmarks
* Lazy import of keystoneclient in osclients
* Add a --uuids-only option to rally task list
* Load OpenStack clients only if those are used
* Modifies VMScenario.\_run\_command function, to handle single bash cmd
* [Sahara] Add support for anti-affinity
* [Sahara] Improved provisioning plugin support
* Adds uninstall\_tempest to API
* Fixes Tempest path update to use abspath
* Fix 'rally task results' output for running task
* Fixed typo in rally/benchmark/validation.py#L415
* Add keystone create\_and\_delete\_service scenario
* Fix rally task start output format
* Add the ability to specify versions for clients
* Add boot server scenario for EC2 API
* [spec] Proposal for new task input file format
* Updated from global requirements
* Fix permissions issue in docker image
* Show precision of 2 in coverage job
* Add keystone get\_entities scenario
* [Tempest]: Try to find py27 interpreter
* Removes from tox.ini flake configuration \_ as builtin
* [Ceilometer] Add CeilometerAlarms.create\_alarm\_and\_get\_history scenario
* Added 'alias' decorator
* [Tempest] add cleanup for clone method
* Fix some typos in comments/doc strings
* Improve hacking and remove useless unit tests
* Add a maximum concurrency option to rps runner
* [unblock-gates] Reduce cirteria of success for Neutron
* Move create-update-delete-stack-inplace test

0.0.2
-----

* Release 0.0.2
* Use get\_client when creating ceilometer client instance
* Fix texts in docs
* [Tempest]: Handle errors from \`requests\` in ConfigGen
* Updated from global requirements
* Add murano client to osclient
* Updated from global requirements
* Add Python 3.4 as supported interpreters
* Fix texts and images in docs
* Fix printing too much output for show secgroup
* Fix bash autocompletion of "rally task sla\_check"
* Introduce in-tree Rally specs
* Unify RPS and constant runners and tune constant runner
* Implement create\_and\_upload\_volume\_to\_image scenario for Cinder
* Fix unstable job
* Unblock gates
* Add create-check-delete stack scenario
* Add create-suspend-resume-delete stack scenario
* [Sahara] Fix Job Binaries download
* [Sahara] cleanup clusters correctly
* Add scenarios for live migration of an instance with volumes
* Mark 'rally use {deployment|task|verification}' deprecated
* Add security group for neutron ports
* [Sahara] added a cluster delete timeout
* Updated from global requirements
* Fix iterate\_per\_tenants and adds docstring
* Add the context custom\_image
* Refactor ImageGenerator and validation
* Adds Designate server-list scenario
* Use \`DEFAULT\_CONFIG' to specify context's config
* Default DNS servers for servers in Neutron network
* \*\*kwargs in create\_image\_and\_boot\_instances only for server
* Fix db configuration in rally\_install.sh
* Updated from global requirements
* Add warnings to not use Tempest benchmarks against production cloud
* Add base for Swift API Benchmarks - Patch-1
* Removed \*\*kwargs for volume in create\_and\_attach\_volume
* Optional arguments for boot\_and\_delete\_server\_with\_secgroups
* Creating OS clients from the environment variables
* Sets new users' endpoint\_type to default to admin's
* Remove all non-incubated oslo code
* Adds Nova keypair tests
* Updated from global requirements
* Updated from global requirements
* Supress warning in cli methods that have sensetive output
* Add new script for gate jobs
* Fix security group setup when users\_per\_tenant is > 1
* Add list-stack-with-resources scenario
* Add few words in docs about installing docker
* More verbose WrongStatusException
* Add user-friendly message if the task cannot be deleted
* Updated from global requirements
* Add context that pre-creates heat stacks
* Add create and delete routers scenarios for Neutron
* Add floating-IP aware VM boot/delete utils
* Prefixing private atomic actions with \`\_\`
* Change error message in secgroups scenario
* Abort scenario execution on SLA failure
* Updated from global requirements
* Fix rally task detailed --iterations-data
* Store keypair and secgroup data on per-user basis
* Mention Rally ReadTheDocs in README
* Don't create task db record during validation
* Add update + add/delete resource cases
* Add "update-inplace/replace resource" cases
* Don't launch stress cleanup in TempestContext
* Add create-update-delete stack scenario
* Support scenarios for updating neutron quotas
* Fix get\_floating\_ip lookup for {"id": net\_id"}
* Looking for floating IP in all networks
* Publishing functional tests reports
* Make unstable-neutron job more stable
* Adds support for multiple tempest sources
* Fix issues in Mistral code
* Implement abort in scenario runners
* Add Mistral benchmark
* Shifting .\_generate\_random\_name() call
* Replace ' with " in rally/benchmark and tests/unit
* Replace ' with " in tests/unit/verification
* Replace ' with " in tests/unit/[aas,cmd]
* Fix template paths in heat samples
* Add option to keep db settings in functional tests
* Replace ' with " in tests/unit/[common,deploy,doc,fixtures]
* Switch to the new Rally API
* Refactor rally.api
* Adds Designate server tests
* Neutron: remove ports before removing network
* Add aliases "rally task|deployment|verify use"
* Adds "scenario" to available verify sets
* Add plugin base
* Add a Python context that logs exceptions
* Replace ' with " in tests/unit/benchmark
* Replace ' with " in [doc, samples]
* Unblock Rally gates
* Replacing ' with " in rally/cmd
* Replace ' with " in rally/[objects,ui,verification]
* Log resource details on TimeoutException
* Replace ' with " in rally/deploy
* Adds postgresql to install\_rally
* Disable security group for some neutron driver
* Replace deprecated max\_failure\_precent sla

0.0.1
-----

* Fix all py3 related issues
* Fix cinder scenario: create nested snapshots
* Correcting minor typos
* Refactor oslo.foo as oslo\_foo
* Updated from global requirements
* Move tempest install to rally.api
* Allow setting DB for unittests via env & add psql support
* Replace ' with " in rally/benchmark
* Replace ' with " in rally/common
* Updated from global requirements
* Add cinder scenario: create volume from volume(clone)
* Add image arg for creating volume from image
* Allow using builtin functions in task templates
* Avoid mapping non-boot volume to boot device
* Improving error message
* Update location to samples
* [Sahara] Add Cloudera plugin support
* Refactoring benchmark/validation.py
* Add cold migrate scenario for Nova test
* Add extend\_volume scenario for Cinder test
* Add list\_snapshots scenario for Cinder test
* Removing "failed" column entirely from rally cli as well as db
* Adds Tempest source repo support for "Tempest" context
* Re-enabling Heat/Neutron resources test
* Allow user to choose Tempest remote repo
* Restructure Rally docs
* Improvements for scenario VMTasks.boot\_runcommand\_delete
* Make Rally gates more stable by moving things to unstable job
* Removes duplicate for generate\_random\_name
* Add unstable job file + return SLA for boot from volume
* Adding duration header to rally verify list query
* Replace ' with " in rally/\*.py
* Moving rally/doc/samples to rally/samples
* Added support to password auth with key pair
* Unblock Rally gates cause by nova bug 1411525
* Adding duration header
* Created new single vm workload sample script using linpack
* Fix "regex" switch for verify
* Input task templates and task cmd cleanup
* Neutron network context support
* Adding dict.items() concatenate validation
* Pulls up \_test\_atomic\_action\_timer to TestCase
* Updated from global requirements
* Optional parameter "endpoint" is added to deployment configuration
* Adding documentation
* Replace ' with " in aas/rest
* Use proper 'open' mocking in unit tests
* Changes Timestamp to hold float instead of datetime
* Adds timestamps to scenarios iterations
* fix devstack URL
* Move files to common lib(Part 4)
* Add list\_volumes scenario for Cinder test
* Updated from global requirements
* Move JSON\_SCHEMA to rally.consts
* Updated from global requirements
* Use json.dumps builtin sorting during plot
* Support generation of report from N task
* Don't hard set keystone endpoint
* Python 3 fix mocking
* Skipping CliUtilsTestCase
* Python 3 fix '+' operand in dict in test\_osclients.py
* Python 3 fix '+' operand in dict\_items
* Fix py3k issues
* Add the flavor context
* use function next() instead of next() method on iterator objects
* Refactoring structure of rally/verification
* Python 3 maxint compatibility
* Wrongly formatted input task
* Adds support for instances with IPv6 addresses in VMScenario
* Move files to common lib(Part 3)
* [Sahara] refactoring \_launch\_cluster
* Use string.ascii\_lowercase and string.ascii\_uppercase only
* Move CinderVolumes tasks from rally.yaml to rally-neutron.yaml
* [Sahara] make job executions delete syncronized
* Use six.moves.range instead of xrange
* Moved module "uuidutils" from oslo-incubator to oslo.utils
* Use six.moves.map instead of itertools.imap
* Use from six.moves.urllib.parse instead of urlparse
* Fix router GW creation
* Refactor validation (part 1)
* syntax change for py3 incompatible
* Add Dockerfile
* Use six.moves.StringIO instead of StringIO.StringIO
* Use six.string\_types instead of basestring
* Move files to common lib(Part 2)
* Move files to common lib(Part 1)
* Fix docstring checker
* Cover Rally with docstrings & Test this coverage
* DataBase: Delete BigText class
* Adding iteritems validation
* Sahara auto\_security\_group parameter added
* Use Network Context in security groups scenario
* Add Network Context Class
* Add list\_servers scenario for Nova
* Remove SLA checks for NovaServers.boot\_server\_from\_volume\_and\_delete
* Updates in Context order
* Fix cli tests vs MOS clouds
* Converting scenario context to dict
* Python 3 incompatible issue: iteritems
* Fix html rendering on failed functional tests
* Add swift service to pre-defined service list
* orchestrator refactor
* Adds Zaqar consumer-producer scenario
* Context tenants refactoring
* Small refactor in run function
* Changing default new user password
* 100% coverage in functional tests
* Updated from global requirements
* Add 'Stop scenario after several errors' feature req
* Configure gate-rally-dsvm-verify
* Fix nested snapshot job
* Updated from global requirements
* Change TaskNotFound traceback info to user-friendly message
* Fix the world (heat role was remove from devstack)
* Use HTTPS to fetch external js resources
* Add cinder stress scenario: nested snapshots
* Updated from global requirements
* Incorrect work rally info find for SLA
* Add scenario for booting vm with security groups
* Adds trove python client to rally.osclients
* Workflow documentation is now in infra-manual
* Updated from global requirements
* Added hacking rule for assertEqual(A in/not in B, True/False)
* add 'rally task list' filters
* Fix rally deployment check fails with https auth\_url
* Further improvements in rally info
* Fix rps runner to use random users from context
* Fix functional test doc
* Add several new hacking rules
* rally info: smart substitutions in the non-ambiguous case
* Reduce amount of time in gates
* Add benchmark overview page to html report
* Add scenarios for security group
* Introudce benchmark.context() decorator
* Adds Nova vm live migration benchmark scenarios
* Engine changed to not crash on context creation
* Add a servers context and a cinder stress scenario
* Add aliases for debug levels in rally.log
* Fix typo in rally tempest config generator
* Add force\_delete to safely delete created VMs
* Improvements for HTML output of rally verify results
* Remove redunant BenchmarkEngine.bind() method
* Refactoring: Chaning atomic ation usage in \_resource\_delete
* Refactoring: removing unused parameter \*\*kwards
* Rename rally-scenarios to rally-jobs & add readme
* TEMPEST: refactor error handling while installation
* Improve unit tests for UserGenerator
* Modify sla failure rate
* Updated from global requirements
* Fix ZeroDivision when no results
* Fix cli arguments backward compatibility
* Fix problem with compare algorithm
* Updated from global requirements
* Changes in getting image id from image name functionality
* Fix invalid Ceilometer task samples
* add ability to refer deployment by uuid or name
* Fix defect of 'separate log level for Rally'
* Port to use oslo.config.fixture
* Fix issue with Pre-Grizzly deployment
* Updated from global requirements
* Base HTML templates and improvements for task report
* Remove check of verification smoke tests
* Sync oslo-incubator
* Fix Zaqar's cleanup
* Added separate log level for Rally debugging
* Updated from global requirements
* Fix typo in Ceilometer's class in the cleanup
* Refactor generic cleanup mechanism part 3
* Updated from global requirements
* Refactor generic cleanup mechanism part 2
* Refactor UserGenerator
* Fix tempest.conf subnets configuration
* Refactor generic cleanup mechanism part 1
* Add SLA to Designate scenarios
* Removing redundancy in (a==b) and 1 or 0
* Updated from global requirements
* Updated from global requirements
* Fix 2 typos and 2 invalid links in doc
* Sahara Cluster error state handling
* Enhance rally info
* Fix rally plugins
* Improve HTML report
* Modify rally commands deployment, task, and verify
* Adds Zaqar Basic scenarios
* Simplify and improve "rally info"
* Fixes fakes and adds version param
* Jobs with scaling benchmark
* Sahara Cluster scaling benchmark
* Add ability to compare two verification results
* Updated from global requirements
* Add functional tests for rally verify
* Refactor tempest verification and tests for it
* Add simple multi thread broker
* Improve plugins and apply loading for runners and contexts plugins
* Port to use oslo.i18n
* Port to use oslo.serialization
* Fix wrong argument in rally verify
* Add detailed description for rally commands
* Adding logic to clean up old rally installs
* Allow user specified tempest.conf
* Remove the obsolete tests\_ci file
* Fix TempestConfig generator keystone API v3 issue
* Using Neutron names in Sahara
* Port to use oslo.utils
* Fix test\_\_generate\_quota\_values\_\* fails
* Updated from global requirements
* Make full bash-completion
* Adding new scenarious for Heat
* Improvements for task plot2html
* Implement max\_avg\_duration criteria
* Fix CONFIG\_SCHEMA in IterationTime in sla/base.py
* Updated from global requirements
* Fix bash completition setup
* Display proper missing argument name
* Fix messages in rally exceptions
* Add misspellings handling to "rally info"
* Improve the deployment's creation and deletion
* Adds Neutron update Port and router benchmarks:
* Fix error status for get\_from\_manager method
* don't call client.authenticate() only when it is authenticated
* Updated from global requirements
* Reorganize test module structure
* Change default endpoint\_type to "public"
* Sahara plugins support
* Add volume option to boot server from volume and run script
* Expand ~ in file paths, specified in rally commands
* Remove run\_tests.sh
* Improve CLI functional tests
* Call correct method to reset quotas
* Catch IOError in cliutils
* Add config file for rally-designate job
* Allow user specified endpoint\_type in config
* Remove two unused exceptions
* Fixes InvalidBenchmarkConfig error while running tempest scenario
* Stop using intersphinx
* Added cleanup for Zaqar objects
* Adds Neutron update Network and subnets benchmarks:
* Sahara Job sequence benchmark added
* Adding optional parameters to Sahara cluster
* Fix bug in atomic actions
* User story - booting 400 servers on real cluster
* Add unit tests for rally.benchmark.scenarios
* Updated from global requirements
* Fix percentile in sla check message
* Remove extraneous vim editor configuration comments
* Fix AttributeError in admin\_cleanup
* add bash completion script
* Store sla result to db
* Add task file validate command
* Adds the Zaqar client
* Add atomic action names tracking
* Add unit tests for rally.benchmark.context
* update rally.conf.sample
* Updated from global requirements
* Fix "rally info" failure in case of empty docstrings
* Rally cli functional testing
* Add hacking checks
* Fix ping\_ip\_address method in order to be run under BSDs
* Add @validation.require\_openstack() validator
* Moved preprocess function from base to types
* Add nova scenarios to rally-neutron.yaml
* Add a cinder volume attachment scenario
* Sahara Job Execution benchmark
* Sahara Cluster and EDP contexts
* User\_stories and feature requests appended to sphinx
* Fix "Resource" typo in debug output
* Fix some typos and mistakes in the doc string
* Updated from global requirements
* Fix CIDR overlapping in \_generate\_subnet\_cidr
* Adds unit tests for rally.cmd.cliutils
* Updated from global requirements
* Minor modification of the \`rally info find\` command
* Allow user specified tempest config file
* Fix syntax errors in JSON doc samples
* Fix disassociation of nova-networks from tenants
* Sahara Job Binary internals cleanup
* Removed admin\_user from benchmark.runners.base
* Fix one more time rally perf job
* Fix rally job in projects that don't have rally-scenario/plugins dir
* Updated from global requirements
* Moved context creation to benchmark.engine
* Rename doc environment to docs
* Make rally perf job work in other projects
* Fix rally verify fail (cause of rally KeyError: 'admin')
* Improve Rally gate job
* Rally info: deploy engines & server providers
* Support for benchmarking with existing users (part 1)
* Add cinder snapshot scenario tests
* Use https for all script sources
* Refactor searchutils module to base entity classes
* Fix rps.\_worker\_process
* Sahara clusters networking parameters
* Designate Records scenarios and tests
* Updated from global requirements
* Improve usability of base.Scenario.validate()
* Impove benchmark validation and cleanup benchmarks decorators
* Designate - update example quotas in scenarios
* Fix wrong parameter used by raise of InvalidBenchmarkConfig
* Updated documentation of "DummyProvider to "ExistingServers"
* Add support for setting up Designate quotas
* Pass named arguments to users.create
* Trying to ingore races related to Nova + restoring missing benchmark
* Mention 'rally info' command in docs
* Add feature\_request mechanism
* Add designate support with domain scenarios
* Fix test\_rps.test\_\_worker\_process test
* Updated from global requirements
* Fix failing tests in gates
* Add new benchmark type VolumeType
* Refactoring rps runner unit tests
* Updated keystone authenticate user\_story
* Updated from global requirements
* Add "rally info" command
* Adding http\_requests scenario
* Disassociate Nova networks during cleanup
* Fix semantic validation of context images
* Decrease jobs time in gates
* Do not assume networks should be assigned
* Update the scenario\_base to base in benchmarks
* Refactor benchmark/context/quota.py
* Workaround for different floating\_ip\_pool results
* Added Sahara Clusters scenario
* Remove magic() method from context manager
* Add required\_contexts validator
* Fix partial results sending in rps runner
* Modify config semantic validation of benchmark engine
* Removed concurency from Quotas.nova\_update\_and\_delete
* Fixes the check for admin role to be case-insenstive
* Fix rally-gate.sh exit status
* Fix copyright in doc/source/config.py
* Split user and admin cleanup to 2 separate classes
* Adds --json,--pprint flags to cmd
* All scenario runs time measurement
* Amended sphinx documents
* Add cinder-update-and-delete quotas scenario
* Extend sla output result
* Refactor Move atomic actions from benchmark scenarios
* Remove openstack-rally entry point
* Update rally neutronclient ca\_cert option
* Mock open to prevent file creation
* Updated from global requirements
* Adds deleting scenarios for Neutron:
* Renames \*.rst files in /rally/doc/samples/tasks
* Adding sla\_check to gate jobs
* Add db model and api for worker
* Added Sahara Image context
* Added Sahara Node Group Templates scenarios
* Fix side menu depth
* Updated from global requirements
* Add resize-server for nova benchmark
* Add support for Keystone API v3
* Adds html result generator for tempest runs
* Modify wrong comment in existing.py
* Periodic runner refactoring
* Updated \_boot\_server() to handle floating ip/external networks
* Refactored test\_images.py and images.py
* Updated rally gate scenarios
* Updated from global requirements
* Set any\_order to True in some unit tests
* Fix cleaning up neutron resources
* Added cleanup for sahara objects
* Reduce number of points in generated charts
* Remove use of random uuid from unit tests
* fixes TypeError coercing of tempfile in subunit2json.py
* Update Tempest Config Generator
* Added Sahara client
* Update the way hard & soft reboots are displayed in task result
* Handle keyboard interrupt in task run
* Revert "Fix failing unit tests"
* Add unit tests for vm benchmark scenarios
* Add basic tools for integration tests
* Add new "volumes" context
* Add support for images context semantic validation
* Add rally task detailed info to rally-plot dir
* Updated test cases in benchmark types
* Updated mock return values to decorators
* Add unit tests for benchmark validation
* Do not use mktemp for creating rally.conf
* Sort by created at column in cli output
* Fix failing unit tests
* Add unit tests for benchmark scenarios
* Add unit tests for tempest context
* Add unit tests for processing utils
* Scenario output should have proper format in case of timeout
* Asynchronous result yielding for scenario runners
* Patched version of hacking library
* Add json results to the logs bundle, as well as html
* Check if required services are available before starting the scenario
* Add tempest unit test cases
* Updated from global requirements
* Change the default value of use\_public\_urls
* Add nova floating ip management in VM scenario
* Ensure temporary results\_dir is created
* Add abstract layer for benchmark context semantic validation
* Check python>=2.7 before running tempest
* Updated from global requirements
* Change the default value of scenario\_output
* Fix creation of ICMP security group rule when using Neutron
* Fix wrong attribute name used for http status
* Add sla checking
* A test to verify that scenarios in rally-scenarios are correct
* Use wget instead of curl in install script
* Updated from global requirements
* Restructure doc/samples/tasks directory
* Added a test to check json<->yaml configs equality
* Delete commented codes from test\_vmtasks.py
* Adding Image Context Class
* Improve doc/source
* use oslo.db module
* small fixes on rest/base.py
* Add Neutron benchmarks scenario for creating and listing ports
* Benchmark to validate a keystone token N times at service endpoint
* Sync necessary modules from oslo incubator for worker service
* Updated from global requirements
* Add unit tests for rest
* Refactor tempest config generator
* Changes tempest subunit log handling
* Minor fixes in Ceilometer benchmark scenarios
* Updated from global requirements
* Fix hacking 0.9.x issues
* Update Rally gates to support VM scenarios
* Add Neutron benchmarks scenario for creating and listing routers
* Rename api directory to rest
* Make quotas context cleanup lazy
* Fixes wrong indent in rally/objects/task.py
* Fixes typo error in Rally - orchestrator/api.py
* Renaming in validation system
* Correct scenario specific results table headers
* Use import from six.moves to import the httplib module
* Removed usage of the py3kcompat module
* Fix wrong result data for constant\_duration runner
* Added user\_stories section
* Bugfix of bug causing security groups from allow\_ssh to remain
* Fixes typo error in rally/README.rst
* fix installation failure if pip is not installed yet
* Add error handling to install\_rally.sh
* Auto install tempest during "rally verify"
* Prepare documentation for readthedocs
* Use FakeCeilometerClient instead of mocks
* Add result table to atomic actions in html
* Add neutron scenario
* Add missing keys to rally runner result
* Heat rally scenario tests
* Add benchmark scenarios for ceilometer queries API
* Fix typos and pep257
* Log deployment id and benchmark key from task
* pass id or name of resources in scenario config
* Ensure file handle cleanup during image creation
* Add scenario name to Scenario Configuration for html
* corresponding result with name in output html
* sorted benchmark scenario selection
* Increase http client timeout from 30s to 180s
* Add benchmark scenarios for ceilometer statistics api
* Add FuelEngine
* Fix typos in file names for ceilometer resources API
* Use plugins from project in rally-gate
* Use function "print\_list" to output tables in rally cmd
* Modify install\_rally.sh to install under BSDs
* Fix inconsistencies in docs for factory classes
* Added a class to measure the duration of atomic operations
* The configs in osclients.py are not in rally.conf.sample
* Add output message for rally task command if uuid is not specified
* Add documentation for plugins
* Unit tests for TempestConf
* replaced e.message
* Add benchmark scenarios for ceilometer resources GET API
* Add benchmark scenarios for ceilometer meters GET API
* Refactoring for random names generation in scenarios
* Enable plots with no successful iterations
* Add probabliity based dummy exception scenario
* Add command-line integrated tests
* Fix typo, rename libxslt1 to libxslt
* Make cleanup methods lazy
* Invalid result for unexisting task in rally task results
* Add Neutron benchmarks scenario for creating and listing subnets
* Add benchmark for tempest. Part 2
* debug level logs should not be translated
* add possibility of using public\_urls for benchmarking
* Skip non yaml/json files when validating samples
* Correction of tempest verification
* Rename number validator parameter none\_ok
* Fix check for tempest tests name
* Add benchmark scenarios for ceilometer alarms api
* Replace non-documented ThreadPool with multiprocessing.Pool
* Fixed issue Unit Test requested Image from Internet
* Add dev libs for xml2 and xslt to install\_rally.sh
* Add Neutron benchmarks scenarios
* Modify the CLI output for 'task detailed'
* Updated from global requirements
* Add mock.patch.stopall() to rally test cleanup
* Sync code from oslo incubator
* Load plugins for scenarios
* Refactored cleaner and added cleanup/utils
* Add argument to curl for following redirects
* Integrate the new oslotest library
* Minor fixes in the visualization of histograms
* Allow to provide a list of uuids for rally task delete
* Don't run rally in debug mode in ci
* Replace non-documented ThreadPool with multiprocessing.Pool
* Add Ironic client
* Adding Dummy benchmark scenarios
* Enlarge Scenario Selector Label
* Create utils module for processing task results
* Pass region\_name to neutron client for consistency
* Make a bigger column for storing task results in mysql
* Runner Type as consts
* Add scenarios directory for ci
* Add new "role" context
* Quotas scenario for nova/cinder
* Use "https" instead of "git" for cloning code
* Add benchmark for tempest. Part 1
* Improve User context
* Name Change for "active\_users" to "concurrency"
* Updated from global requirements
* Renamed provider 'name' property to 'type'
* Fix deploy engine "name" property in a docstring
* Fix DevStack contrib
* Clean rally ENV after recreate database
* Changed all deploy engine's 'name' property to 'type'
* Columns name change for ScenarioRunnerResult
* Fix ubuntu-12 issues in LxcHost
* Updated from global requirements
* Fix broken Glance cleanup context
* Allow to call task delete without specifying --uuid parameter
* Fix column index for sorting in rall verify list
* Add LxcEngine
* Fix cleanup in LxcProvider
* move cleaner context users/admin into setup()
* Create vm benchmark module
* Add number and file validators
* Remove createfromenv test environment dependency
* Improve tempest.conf generation
* Split ContinuousScenarioRunner to constant and constant duration
* Region support
* Replace inline schema urls with reference to single piece of data
* Add multihost engine
* Add neutron quotas management in benchmark context
* Added create\_and\_list\_users and create\_and\_list\_tenants benchmark
* Add --filename to rally task start
* Create class "Connection" for sqlalchemy.api
* Add required\_parameters validator
* Adds commands for verification results
* Adds saver for results from \`rally verify\`
* Updated from global requirements
* Skip large-ops benchmarks
* Add histogram to the report of tasks' results
* move install script to project root dir
* Remove file AUTHORS
* Re-Add default db connection string
* Improve WaitForTestCase
* Remove \_change\_port
* Add task uuid to iteration start/end logs
* Add Scenario meta() to extract meta information from method
* There is no mock\_magic.assert\_called\_once method
* Create copy of scenario context before modifying
* Add benchmark-context manager
* Improve @base.scenario to specify it is admin only
* non-zero exit status on error for CLI
* Add support for the neutron client in the osclients object
* remove openstack-patches
* Sync common db code from Oslo
* Use pool of threads in UserGenerator for deleting tenants and users
* Add tenants create, with users per tenant benchmark for keystone
* Use float values in cfg.FloatOpt() options
* Add nova/cinder quotas management in benchmark context
* add +x to pre\_test\_hook.sh
* Add @cleanup usage in each scenario method
* Add tests\_ci/pre\_test\_hook.sh
* Add support to upload image from a local directory
* Fix default dummy engine name in devstack plugin
* Fix dependencies for rpm based systems
* Add execrise script to devstack
* Simple fix for bug with periodic runner
* add create\_and\_list\_image benchmark
* Pass context to secnarios
* Moved fuelclient.py
* Fix samples in README.rst
* Improve generic cleanup performance
* Add a ScenarioRunner subclass which executes scenarios serially
* Use pool of threads in UserGenerator for creating tenants and users
* Sync modules from oslo, excluding db related
* decorator for benchmark scenarios method
* Fix Nova crap
* Add cliutils.args for task\_id in use task command
* Rename deploy\_id argument to uuid in deployment commands
* Rename task\_id argument to uuid in task commands
* Updated from global requirements
* Add default task\_id management by using @default\_from\_global
* Use testr directly in tempest verifier
* rename DummyProvider to ExistingServers
* rename DummyEngine to ExistingCloud
* Refactor text in use deployment command
* Fix envutils.get\_global (prev. default\_deployment\_id) errors management
* Add fix \_\_ctx\_name\_\_ in keypair context
* add create\_and\_list\_volume benchmark
* Add shell script for integrated tests
* show detailed exception when doing cleanup
* remove a mixed style (\*\*{} and param=value)
* Add boot\_and\_list\_server benchmark
* Use ConfigParser to generate tempest config
* standardized output for scenario runners
* Updated from global requirements
* Refactor Tempest installation directories structure
* unify log\_task\_wrapper and log\_deploy\_warpper
* Improve OpenstackProvider
* Fix CONFIG\_SCHEMA for runners
* Move serverprovider under deploy
* Base part of a server REST API
* Added tenant create benchmark for keystone
* remove unused code in serverprovider and tests
* improve rally output (exception part)
* Remove underscore (\_use) import in deployment command
* Remove base.Scenario.register()
* Refactor the decorator which sets a default deploy\_id
* New task config and verification refactoring
* Add print openrc information for use deployment
* Add unit test for deployment list subcommand
* print all debug msg from clients when using debug flag
* remove some unused codes
* Validator for image checking
* Use itertools instead of custom count
* Use six.add\_metaclass instead of \_\_metaclass\_\_
* Add unit test for Authenticate scenario
* Make tempest integration works without "sudo"
* Show atomic operations results without aggregation with CLI
* Add keypair context
* Fix for Authentication scenario to correctly use self.clients
* Capture the pkey errors
* Switch to oslosphinx
* Don't raise exception if deploy\_id not found
* Fix typo in deployment check
* Set 'period', 'duration' config parameters in seconds
* Add the install\_rally.sh file
* Expose clients directly from osclients.Clients
* Updated from global requirements
* Make usage serverprovider in common way
* Updated from global requirements
* catch exception when deployment not found
* Remove unnecessary flush in task detailed
* exclude INFO log about "Starting new HTTP connection..."
* move wait\_for & wait\_for\_delete into benchmark.utils
* Added "active" column on deployment list
* remove keypair from secgroup context
* Add git branches support in DevstackEngine
* Fix cleanup in DevstackEngine
* Add count to atomic actions display
* Make lxc ssh port fowarding optional
* don't delete network during benchmark
* Unify Context classes and introduce context object
* added decorator-based mocks declaration to
* added decorator-based mocks declaration to
* ignore build dir in tox.ini
* use CONF in glance benchmark
* Updated from global requirements
* Add unit tests for the show command
* Add 'flavor' to fakes.py
* Changed import statement
* Add support to tag a task
* Removes old obsolete task.plot function and all related code
* Fix user creation in devstack engine
* Fix and unify structure of modules in benchmark engine
* Fix ScenarioRunnerTestCase:test\_run\_scenario\_timeout
* Prepare for real benchmark context support
* Proper fix for test\_run\_scenario\_exception\_inside\_test
* Remove cinder volume type deletion
* Fix CIDR key error bug
* Move scenario args validation to BenchmarkEngine
* Add use task subcommand
* Fix test\_run\_scenario\_exception\_inside\_test
* Add deploy-id to command 'use deployment'
* Updated from global requirements
* Refactor use deployment command
* Improve CLI: rally task list (of empty sequence)
* Improve CLI: rally deployment list (of empty sequence)
* Add tests for utils.\_prepare\_for\_instance\_ssh
* Add yaml file support
* Add unit test for use deployment with a deployment\_id not found
* Remove unused methods from rally.benchmark.uitls
* Finish unit tests for ScenarioRunner
* Improve benchmark.runner.UserGenerator
* Updates \_cleanup\_users\_resources to use imap instead of map
* Add v3 uri to tempest configuration
* Fix typo in use deployment subcommand
* Add ceilometer python client to rally.osclients
* Improve README.rst
* Add a new rally command: show
* Add task result graphic ploter command
* Updated from global requirements
* Rally & Tempest integration (base) - part 2
* Updated from global requirements
* UPDATED README to make it detailed
* UPDATED README to make it detailed
* Do not use openstack clients as a thread safe objects
* Use Endpoint objects instead of endpoint dictionaries
* Show 90 and 95 percentile results
* Print exceptions from scenarios only in case of debug
* Add permission to validators
* Fix relative path in Tempest verification UTs
* Import missing gettextutils
* Rename \_run\_scenario\_loop --> \_run\_scenario\_once
* Rally & Tempest integration (base) - part 1
* Rename TestEngine --> BenchmarkEngine
* Add support for user/admin permissions in endpoints
* Move every rally command to its own module
* Fix openstack server provider resource cleanup
* Added SSL support for OpenStack services clients
* Validate flavor\_id in glance benchmark
* Added Heat client initialization
* Fix mocking in scenario runner tests
* Update OS serverprovider schema to allow specifying image by uuid only
* Replace the call to default\_deployment\_id() by a decorator
* Add ssh port forwarding to lxc provider
* Show whether task failed when printing detailed information
* Implement ScenarioRunner execution types via inheritance
* Fix rally benchmark's timeout mechanism
* Fix py26 timeouts
* Do not cache not connected ssh client
* Use new deployemnts by default, add --no-use option
* Add atomic actions timer for glance scenarios
* Don't initalise deployment engine when starting a task
* Don't require glance to use OpenStack server provider
* Add authenticate benchmark for Keystone
* Fix misspellings in rally
* Add benchmark tests for glance
* Remove magic numbers from benchmark utils
* Mark task as failed if benchmark validation fails
* Use assertIsInstance to test the results of get\_engine and get\_provider
* Start refactoring scenario runner
* Use task.set\_failed when task is failed
* Add display of atomic actions timer in CLI
* Add rally use command - part 2
* Sshutils refactoring
* Fix benchmark.utils.get\_from\_manager()
* Add timer for atomic actions
* Fix idle time in timeout case
* Validator for flavor\_id
* Temporary fix testtols version to unblock gate
* Validator for image\_id
* Move cleanup functions to benchmark.utils
* Call rally-manage with warning when openstack-rally-manage is used
* Add deployment check subcommand
* Validators adder for benchmarks preconditions
* Add FuelClient
* Add option to use deployment on creation
* Create rally deployment from OS\_\* environment variables
* Remove debug print from fileutils
* Add name into dummy engine schema and make it required
* Add rally use command
* LxcProvider network refactoring
* Fix tests/benchmark/scenarios/nova/test\_utils.py
* Use cls instances instead of object in benchmarks
* Add keystone host unreachable checks
* Print task information when creating task from CLI
* Benchmark scenarios for Rally: run task in instance
* Improve devstack install script
* Add management of unauthorized keystone exceptions
* Added missing test for rally.cmd.main.task.results
* Fix CLI output on task detailed command
* Introduce the Endpoint object
* Print deployment id when a deployment is created
* Refactor unit tests for dummy engine
* Remove results before a task
* Extra scripts with DevStack support
* Omit openstack/common/ in test coverage reports
* Add support to build docs by Sphinx
* Add a separate class for a mutable JSONEncodedDict
* Improve tests of DevstackEngine.deploy
* Clarify usage of an endpoint
* Deprecate openstack- console scripts, introduce rally, rally-manage
* Rename Server's 'ip' to 'host'
* Add boot\_from\_volume benchmark
* Fix icmp security group rule creation
* Fix devstack engine test coverage
* Check endpoints with keystone client
* Add missing tests for rally/benchmark/scenarios/nova/utils.py
* DummyProvider refactoring
* Create FK for parent\_uuid on ALTER TABLE
* Switch rally.sshutils from 'eventlet' to 'time'
* Move rally.test to tests.test
* Make cli result table columns more clean
* Adding cinder benchmark scenarios for Rally
* Remove nullable=True because it is set by default in db
* Fix comments and typos in db/api, orchestrator/api and cmd/main
* Sync with global requirements
* Fix H302 pep8
* Add base utils for keystone benchmarks
* Move nova benchmark samples to nova directory
* Log warnings instead of infos about cleanup fails
* Make resource status checking letter case independent
* Add generic keystone cleanup mechanism
* Show results of last task without requring user to lookup UUID
* Refactor the way benchmark engine returns its results
* Fix invalid port range in security group rule create
* Prepare instances for SSH access
* Preserve consistency in module namings
* Fix incorrect exit status checking in sshutils
* Remove duplicate mocks from ScenarioTestCase
* Move ScenarioRunner class to a separate module
* Remove mox
* Use hacking import\_exceptions for gettextutils.\_
* Add missing tests for rally/cmd/manage.py
* Fix usage of get() methods in python clients
* Add \`port\` attribute to Server object
* Fix tcp port in sshutils
* Fix ssh port in devstack engine
* Add \`parent\` field to deploy object
* Rename "destroy\_vms" to "destroy\_servers"
* Remove unused sh requirments
* Add .venv to git ignore
* Get rid of fuel-ostf-tests
* Rename "create\_vms" to "create\_servers"
* Update oslo code
* Fix too slow .VirshProviderTestCase.test\_create\_vm
* Add start and finish time of the deployment
* Add ability to get stdout/stderr to sshutils
* Show task details without results
* Removed \`tests' section from config samples
* Implement periodic execution of benchmark scenarios
* Move Fake objects to a special tests.fakes module
* Add CONFIG\_SCHEMA to virsh provider
* Add CONFIG\_SCHEMA to dummy provider
* fix the wrong file name for devstack-in-openstack sample json file
* Benchmark scenarios for Rally: Nova rescue server
* Add cover to gitignore
* Add CONFIG\_SCHEMA to DummyEngine
* Add caching to osclients.Clients
* Cosmetic change to cleanup output
* Separate deployments from tasks
* Fix bug in serverprovider.test\_openstack\_provider
* Store/restore Server from a dict with credentials
* Fix bug in tests.serverprovider.test\_virsh
* LxcProvider track resources
* Store/restore Server from a dict with credentials
* Add CONFIG\_SCHEMA to openstack provider
* Add common config validation to ProviderFactory
* Add CONFIG\_SCHEMA to devstack engine
* Add common config validation to EngineFactory
* Fix bug in tests.benchmark.test\_utils
* Generic benchmark cleanup
* Refactoring for generic benchmark cleanup
* OpenStackProvider track resources
* Remove argument vm\_uuids from destroy\_vms
* VirshProvider track resources
* Quick fix for boot\_and\_bounce\_server
* Cover VirshProvider by unit tests
* Introduce a manager of resources
* Remove a task from providers
* Resource model with DB API
* Rename ServerDTO to Server
* Run benchmark scenarios for a given period of time
* Renamed sample with OpenStack provider
* Separate samples
* Add CLI command for deployments
* Update completed tasks status to "finished"
* Data processing: aggregation plots
* Tracking deployment status into context on error
* Fix bug in tests.benchmark.test\_utils
* Add openstack based server provider
* Separate command to perform deploy and cleanup
* Enable access to admin openstack clients from benchmarks
* Add uuid to temporary tenant names
* Fix bug in ScenarioRunner
* Fix passing the context to multiprocessing pool
* Switch sshutils.SSH from subprocess to paramiko
* Introduce the deployment
* Change benchmark config format
* Refactor nova scenarios benchmark tests
* Proper update of objects
* Make Deploy and Task objects similar
* Add tests for rally.objects.task.Task
* Move task and deploy modules in rally.objects
* Fail task when engine not found
* Remove old sshutils functions
* Fix debian/ubuntu related issues
* Cover rally.orchestrator.api by unit tests
* Fix typo in name of the orchestration test file
* Abstract out DB operations on deploys in db.deploy
* Added Deployment model
* Canonical form of the test\_db\_api.py filename
* Fix incorrect TaskCommandsTestCase.test\_start test
* Add btrfs support to LxcContainer
* LxcProvider refactoring
* Switch LxcProvider from sshutils
* Improve sample configs
* Delete task by command
* Add sleep\_between() method to benchmark scenarios
* Fedora/CentOS specific fixes in devstack engine
* Fixes rally boot\_server benchmark bug
* Benchmark scenario to boot server and bounce it N times
* Delete objects completely
* Optimize the scenario runner
* Fix intermitten py test failure
* Use a dash in --task-id argument
* Use ServerDTO.ssh in devstack engine
* Add ssh object to ServerDTO
* Add SSH class to rally.sshutils
* Cleanup logging
* Rename NoDBTestCase to TestCase
* Enable different tenants in the scenario runner
* Fix local test case failure for test\_lxc\_install()
* Fully initialize the keystone client in osclients
* Use random network in boot\_server scenario
* Performance changes for UTs
* Add lxc provider
* Catch proper exception on MissignArgs error
* Fixed typos into constants
* Improve task results cli cmd
* Adds boot server benchmark test
* Add task to provider object
* Improve rally.utils.log\_task\_wrapper()
* Add utils.log\_task\_wrapper() method
* Work out the sample task config
* Simplify the test config format
* Move fixtures from test-requirments to requirments
* Added python clients patches
* Get rid of rally.benchmark.tests module
* Remove legacy code from test engine
* Fix bug in DevstackEngineTestCase.test\_deploy
* Fix NovaServers.snapshot benchmark
* Fixed some formatting and typos in the README
* Update short description language
* Check for VM ERROR state rather than FAILED
* Make DevstackEngine work out of the box
* update rally conf sample
* Update oslo code
* Fixed typo in DevstackEngine docstring
* Rename FakeEngine DummyEngine
* Fix wrong usage of Pool
* Add new method "raw\_results" to Task CLI
* Cleanup db.TaskResource model
* Remove vim header
* Benchmark scenarios for Rally: Nova servers
* Benchmark scenarios for Rally: Nova utils
* Improve utils.wait\_for()
* Add self to AUTHORS file
* Rewrite rally.utils.sync\_execute() to wait\_for()
* Make devstack engine more flexible
* Store test scenario results in DB
* Fixed default value for ssh public key
* Fixing Rally config json schema
* Make ProviderFactory.get\_provider more simple
* Adding two more options to verification\_tests options
* Fixing incorrect mocks usage in unit tests
* Renaming a test case
* Add DummyProvider
* Various fixes in benchmark.engine
* Fix rally.benchmark.scenarios autoload
* Make rally.utils.import\_modules\_from\_package recursive
* Fix verification output
* Reducing the running time for test\_sync\_execute()
* Rename benchmark.engine.Tester to benchmark.engine.Verifier
* Add new Benchmark engine
* Move rally.task to rally.db.task
* Remove old test engine stuff
* Fix orhestrator.api test
* Fixing decorators for rally.benchmark.base.Scenario
* Add benchmark scenarios base class
* Add Timer to rally.utils
* Slight improvement for utils.sync\_execute
* Removing a needless verification test
* Renaming benchmark folders
* Rally logging: deploy engine
* Function synchronization utility
* Fix logging
* Rally logging: test engine
* Patched virsh serverprovider, devstack deploy engine to use ServerDTO
* Added Devstack deploy engine
* Add Server Provider DTO objects
* Move and refactor ImmutableMixin & EnumMixin to rally.utils
* Fix  verification results are not stored in db
* Added virsh vm provider engine
* Fixed import utils code
* Fix Tester cleanup crashes rally execution
* Move deploy() out of \_\_enter\_\_ method
* Bugfix for TestEngine
* Bump oslo config
* Implement task cli list method
* Rally integration with fuel-ostf-tests
* Removed except blocks from cli code
* Fix bug in CLI task.start method
* Cancel remaining tests if one test was timed out
* Add task status tracking in TestEngine
* Add Task object for tracking task status
* Added cli scripts to package config
* Bugfix for ConfigManagers: defauls overwriting
* Move task creation from get\_engine to orcehstrator
* Fixed incorrect timeout tests
* Change create\_image signature
* Rename vmprovider to serverprovider module
* Add task status tracking
* Adding new test execution layer
* Benchmark test framework for Rally
* Add utils to work with OpenStack python clients
* VMProviderFactory.\*\_image() shouldn't be abstract
* Improve readme
* Added openstack patches for nova
* Config manager
* Partial implement fake deploy engine
* Rename rally.cmd.task\_cli module to rally.cmd.main
* Add CLI utils for DB recreate
* db.db\_create() shouldn't call hiddenly db.db\_drop()
* Fix VM provider get\_available\_providers() test
* Slight refactoring of the testcase for Tester.run()
* Exception raising in TestEngine.verify()
* Test engine redundant code removal
* Import modules from package
* Add .gitreview file
* Test engine & Test cases
* Add CLI interface for Rally
* Add cliutils
* Add version module
* Add orchestrator
* Test engine
* Force tox version to 1.6
* Repaire benchmark test
* Add author and fix .gitignore
* A fix for tox.ini
* Authors info
* Adding cleanup for all test runner calls
* Add VM Provider factory
* Add deploy engine factory
* Add itersubclasses() method that allows to iter through all subclass
* Init benchmark framework
* README for Rally
* Move tests to root dirrectory and fix tox
* Add Task Model
* Add base for DB API
* Add DB utils from Oslo
* Add run\_tests.sh as alternative for tox
* Add venv tools
* Add oslo fixtures
* Fix test-requirements.txt
* Add oslo test module
* Update gettextuils and jsonutils form oslo
* Add rally.sample.conf to project
* Add AUTHORS file
* Add tox, pep and requirements to project
* Rename README.md to README.rst
* Integrate with oslo
* Initial commit
