Skip to content

Commit 4a37da2

Browse files
aholmbergaholmberg
committed
execution profile docs
PYTHON-569
1 parent 4102026 commit 4a37da2

4 files changed

Lines changed: 235 additions & 8 deletions

File tree

‎cassandra/cluster.py‎

Lines changed: 60 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -198,11 +198,49 @@ def default_lbp_factory():
198198

199199
class ExecutionProfile(object):
200200
load_balancing_policy = None
201+
"""
202+
An instance of :class:`.policies.LoadBalancingPolicy` or one of its subclasses.
203+
204+
Used in determining host distance for establishing connections, and routing requests.
205+
206+
Defaults to ``TokenAwarePolicy(DCAwareRoundRobinPolicy())`` if not specified
207+
"""
208+
201209
retry_policy = None
202-
consistency_level = None
210+
"""
211+
An instance of :class:`.policies.RetryPolicy` instance used when :class:`.Statement` objects do not have a
212+
:attr:`~.Statement.retry_policy` explicitly set.
213+
214+
Defaults to :class:`.RetryPolicy` if not specified
215+
"""
216+
217+
consistency_level = ConsistencyLevel.LOCAL_ONE
218+
"""
219+
:class:`.ConsistencyLevel` used when not specified on a :class:`.Statement`.
220+
"""
221+
203222
serial_consistency_level = None
204-
request_timeout = None
205-
row_factory = None
223+
"""
224+
Serial :class:`.ConsistencyLevel` used when not specified on a :class:`.Statement` (for LWT conditional statements).
225+
"""
226+
227+
request_timeout = 10.0
228+
"""
229+
Request timeout used when not overridden in :meth:`.Session.execute`
230+
"""
231+
232+
row_factory = staticmethod(tuple_factory)
233+
"""
234+
A callable to format results, accepting ``(colnames, rows)`` where ``colnames`` is a list of column names, and
235+
``rows`` is a list of tuples, with each tuple representing a row of parsed values.
236+
237+
Some example implementations:
238+
239+
- :func:`cassandra.query.tuple_factory` - return a result row as a tuple
240+
- :func:`cassandra.query.named_tuple_factory` - return a result row as a named tuple
241+
- :func:`cassandra.query.dict_factory` - return a result row as a dict
242+
- :func:`cassandra.query.ordered_dict_factory` - return a result row as an OrderedDict
243+
"""
206244

207245
def __init__(self, load_balancing_policy=None, retry_policy=None,
208246
consistency_level=ConsistencyLevel.LOCAL_ONE, serial_consistency_level=None,
@@ -259,6 +297,12 @@ def default(self):
259297

260298

261299
EXEC_PROFILE_DEFAULT = object()
300+
"""
301+
Key for the ``Cluster`` default execution profile, used when no other profile is selected in
302+
``Session.execute(execution_profile)``.
303+
304+
Use this as the key in ``Cluster(execution_profiles)`` to override the default profile.
305+
"""
262306

263307

264308
class _ConfigMode(object):
@@ -907,6 +951,19 @@ def __init__(self, street, zipcode):
907951
UserType.evict_udt_class(keyspace, user_type)
908952

909953
def add_execution_profile(self, name, profile, pool_wait_timeout=5):
954+
"""
955+
Adds an :class:`.ExecutionProfile` to the cluster. This makes it available for use by ``name`` in :meth:`.Session.execute`
956+
and :meth:`.Session.execute_async`.
957+
958+
Normally profiles will be injected at cluster initialization via ``Cluster(execution_profiles)``. This method
959+
provides a way of adding them dynamically.
960+
961+
Adding a new profile updates the connection pools according to the specified ``load_balancing_policy``. By default,
962+
this method will wait up to five seconds for the pool creation to complete, so the profile can be used immediately
963+
upon return. This behavior can be controlled using ``pool_wait_timeout`` (see
964+
`concurrent.futures.wait <https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.wait>`_
965+
for timeout semantics).
966+
"""
910967
if not isinstance(profile, ExecutionProfile):
911968
raise TypeError("profile must be an instance of ExecutionProfile")
912969
if self._config_mode == _ConfigMode.LEGACY:

‎docs/api/cassandra/cluster.rst‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,8 @@
7474

7575
.. automethod:: unregister_listener
7676

77+
.. automethod:: add_execution_profile
78+
7779
.. automethod:: set_max_requests_per_connection
7880

7981
.. automethod:: get_max_requests_per_connection
@@ -106,6 +108,11 @@
106108

107109
.. automethod:: set_meta_refresh_enabled
108110

111+
.. autoclass:: ExecutionProfile
112+
:members:
113+
114+
.. autodata:: EXEC_PROFILE_DEFAULT
115+
:annotation:
109116

110117
.. autoclass:: Session ()
111118

‎docs/execution_profiles.rst‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
Execution Profiles (experimental)
2+
=================================
3+
4+
Execution profiles are an experimental API aimed at making it easier to execute requests in different ways within
5+
a single connected ``Session``. Execution profiles are being introduced the exploding number of configuration options,
6+
especially as the database platform evolves more complex workloads.
7+
8+
The Execution Profile API is being introduced now, in an experimental capacity in order to take advantage of it in
9+
existing projects, and to guage interest and feedback in the community. For now, the legacy configuration remains
10+
intact, but legacy and Execution Profile APIs cannot be used simultaneously on the same client ``Cluster``.
11+
12+
This document explains how Execution Profiles relate to existing settings, and shows how to use the new profiles for
13+
request execution.
14+
15+
Mapping Legacy Parameters to Profiles
16+
-------------------------------------
17+
18+
Execution profiles can inherit from :class:`.cluster.ExecutionProfile`, and currently provide the following options,
19+
previously input from the noted attributes:
20+
21+
- load_balancing_policy - :attr:`.Cluster.load_balancing_policy`
22+
- request_timeout - :attr:`.Session.default_timeout`, optional :meth:`.Session.execute` parameter
23+
- retry_policy - :attr:`.Cluster.default_retry_policy`, optional :attr:`.Statement.retry_policy` attribute
24+
- consistency_level - :attr:`.Session.default_consistency_level`, optional :attr:`.Statement.consistency_level` attribute
25+
- serial_consistency_level - :attr:`.Session.default_serial_consistency_level`, optional :attr:`.Statement.serial_consistency_level` attribute
26+
- row_factory - :attr:`.Session.row_factory` attribute
27+
28+
When using the new API, these parameters can be defined by instances of :class:`.cluster.ExecutionProfile`.
29+
30+
Using Execution Profiles
31+
------------------------
32+
Default
33+
~~~~~~~
34+
35+
.. code:: python
36+
37+
from cassandra.cluster import Cluster
38+
cluster = Cluster()
39+
session = cluster.connect()
40+
local_query = 'SELECT rpc_address FROM system.local'
41+
for _ in cluster.metadata.all_hosts():
42+
print session.execute(local_query)[0]
43+
44+
45+
.. parsed-literal::
46+
47+
Row(rpc_address='127.0.0.2')
48+
Row(rpc_address='127.0.0.1')
49+
50+
51+
The default execution profile is built from Cluster parameters and default Session attributes. This profile matches existing default
52+
parameters.
53+
54+
Initializing cluster with profiles
55+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
56+
57+
.. code:: python
58+
59+
from cassandra.cluster import ExecutionProfile
60+
from cassandra.policies import WhiteListRoundRobinPolicy
61+
62+
node1_profile = ExecutionProfile(load_balancing_policy=WhiteListRoundRobinPolicy(['127.0.0.1']))
63+
node2_profile = ExecutionProfile(load_balancing_policy=WhiteListRoundRobinPolicy(['127.0.0.2']))
64+
65+
profiles = {'node1': node1_profile, 'node2': node2_profile}
66+
session = Cluster(execution_profiles=profiles).connect()
67+
for _ in cluster.metadata.all_hosts():
68+
print session.execute(local_query, execution_profile='node1')[0]
69+
70+
71+
.. parsed-literal::
72+
73+
Row(rpc_address='127.0.0.1')
74+
Row(rpc_address='127.0.0.1')
75+
76+
77+
.. code:: python
78+
79+
for _ in cluster.metadata.all_hosts():
80+
print session.execute(local_query, execution_profile='node2')[0]
81+
82+
83+
.. parsed-literal::
84+
85+
Row(rpc_address='127.0.0.2')
86+
Row(rpc_address='127.0.0.2')
87+
88+
89+
.. code:: python
90+
91+
for _ in cluster.metadata.all_hosts():
92+
print session.execute(local_query)[0]
93+
94+
95+
.. parsed-literal::
96+
97+
Row(rpc_address='127.0.0.2')
98+
Row(rpc_address='127.0.0.1')
99+
100+
Note that, even when custom profiles are injected, the default ``TokenAwarePolicy(DCAwareRoundRobinPolicy())`` is still
101+
present. To override the default, specify a policy with the :data:`~.cluster.EXEC_PROFILE_DEFAULT` key.
102+
103+
.. code:: python
104+
105+
from cassandra.cluster import EXEC_PROFILE_DEFAULT
106+
profile = ExecutionProfile(request_timeout=30)
107+
cluster = Cluster(execution_profiles={EXEC_PROFILE_DEFAULT: profile})
108+
109+
110+
Adding named profiles
111+
~~~~~~~~~~~~~~~~~~~~~
112+
113+
New profiles can be added constructing from scratch, or deriving from default:
114+
115+
.. code:: python
116+
117+
from cassandra.cluster import ExecutionProfile
118+
from cassandra.policies import WhiteListRoundRobinPolicy
119+
locked_execution = ExecutionProfile()
120+
locked_execution.load_balancing_policy = WhiteListRoundRobinPolicy(['127.0.0.1'])
121+
node1_profile = 'node1_whitelist'
122+
cluster.add_execution_profile(node1_profile, locked_execution)
123+
124+
for _ in cluster.metadata.all_hosts():
125+
print session.execute(local_query, execution_profile=node1_profile)[0]
126+
127+
128+
.. parsed-literal::
129+
130+
Row(rpc_address='127.0.0.1')
131+
Row(rpc_address='127.0.0.1')
132+
133+
See :meth:`.Cluster.add_execution_profile` for details and optional parameters.
134+
135+
Passing a profile instance without mapping
136+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
137+
138+
We also have the ability to pass profile instances to be used for execution, but not added to the mapping:
139+
140+
.. code:: python
141+
142+
from copy import copy
143+
from cassandra.query import tuple_factory
144+
145+
tmp = copy(node1_profile)
146+
tmp.request_timeout = 100
147+
tmp.row_factory = tuple_factory
148+
149+
print session.execute(local_query, execution_profile=tmp)[0]
150+
print session.execute(local_query, execution_profile='node1')[0]
151+
152+
.. parsed-literal::
153+
154+
('127.0.0.1',)
155+
Row(rpc_address='127.0.0.1')
156+
157+
As shown above, the ``tmp`` profile shares a load balancing policy with one managed by the cluster. If this technique
158+
is not used, the application would need to initialize and maintain the policy state manually.
159+

‎docs/index.rst‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Contents
1616
How to install the driver.
1717

1818
:doc:`getting_started`
19-
A guide through the first steps of connecting to Cassandra and executing queries.
19+
A guide through the first steps of connecting to Cassandra and executing queries
2020

2121
:doc:`object_mapper`
2222
Introduction to the integrated object mapper, cqlengine
@@ -25,22 +25,25 @@ Contents
2525
The API documentation.
2626

2727
:doc:`upgrading`
28-
A guide to upgrading versions of the driver.
28+
A guide to upgrading versions of the driver
29+
30+
:doc:`execution_profiles`
31+
An introduction to a more flexible way of configuring request execution
2932

3033
:doc:`performance`
3134
Tips for getting good performance.
3235

3336
:doc:`query_paging`
34-
Notes on paging large query results.
37+
Notes on paging large query results
3538

3639
:doc:`lwt`
3740
Working with results of conditional requests
3841

3942
:doc:`user_defined_types`
40-
Working with Cassandra 2.1's user-defined types.
43+
Working with Cassandra 2.1's user-defined types
4144

4245
:doc:`security`
43-
An overview of the security features of the driver.
46+
An overview of the security features of the driver
4447

4548
:doc:`dates_and_times`
4649
Some discussion on the driver's approach to working with timestamp, date, time types
@@ -55,6 +58,7 @@ Contents
5558
installation
5659
getting_started
5760
upgrading
61+
execution_profiles
5862
performance
5963
query_paging
6064
lwt

0 commit comments

Comments
 (0)