wiredtiger.github.com/develop/cursor_log.html at master · wiredtiger/wiredtiger.github.com

History

210 lines (208 loc) · 21.8 KB

Raw

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<head>

<title>WiredTiger: Log cursors</title>

</head>

<body>

<tbody>

 <span id="projectnumber">Version 12.0.0</span>

</div>

</td>

</tr>

</tbody>

</table>

</div>

<a href="https://github.com/wiredtiger/wiredtiger">Fork me on GitHub</a>

<a class="last" href="http://groups.google.com/group/wiredtiger-users">Join my user group</a>

</div>

/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */

var searchBox = new SearchBox("searchBox", "search",false,'Search');

/* @license-end */

</script>

/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */

$(function() {

initMenu('',true,false,'search.php','Search');

$(document).ready(function() { init_search(); });

});

/* @license-end */</script>

</div>

</div>

<div id="splitbar" style="-moz-user-select:none;"

class="ui-resizable-handle">

</div>

/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */

$(document).ready(function(){initNavTree('cursor_log.html',''); initResizable(); });

/* @license-end */

</script>

<div id="MSearchSelectWindow"

onmouseover="return searchBox.OnSearchSelectShow()"

onmouseout="return searchBox.OnSearchSelectHide()"

onkeydown="return searchBox.OnSearchSelectKey(event)">

</div>

name="MSearchResults" id="MSearchResults">

</iframe>

</div>

<div class="title">Log cursors </div> </div>

</div>

<div class="textblock"><p>WiredTiger cursors provide access to data from a variety of sources, and one of these sources is the records in the transaction log files. Log files may not be present in every WiredTiger database, only databases that have been configured for logging using the <code>log</code> configuration for <a class="el" href="group__wt.html#gacbe8d118f978f5bfc8ccb4c77c9e8813" title="Open a connection to a database.">wiredtiger_open</a>. In databases with log files, a log cursor provides access to the log records. Although log cursors are read-only, applications may store records in the log using <a class="el" href="struct_w_t___s_e_s_s_i_o_n.html#ac85541ca42d7596857d9f50e03c78eb5" title="Insert a WT_LOGREC_MESSAGE type record in the database log files (the database must be configured for...">WT_SESSION::log_printf</a>. While the log cursor is open automatic log file removal, even if enabled, will not reclaim any log files.</p>

<p>Each physical WiredTiger log record represents one or more operations in the database. When a log record represents more than a single operation in the database, all of the operations in a log record will be part of the same transaction. Using a log cursor, walking a transaction commit log record will return each operation of the transaction in the record individually. If a transaction updates both logged and not logged tables, only the operations on the logged tables appear in the record.</p>

<p>The following examples are taken from the complete example program <a class="el" href="ex_log_8c-example.html">ex_log.c</a>.</p>

<p>To open a log cursor on the database:</p>

<div class="fragment"><div class="line"> error_check(session-><a class="code" href="struct_w_t___s_e_s_s_i_o_n.html#afb5b4a69c2c5cafe411b2b04fdc1c75d">open_cursor</a>(session, <span class="stringliteral">"log:"</span>, NULL, NULL, &cursor));</div>

</div><p> A log cursor's key is a unique log record identifier, plus a <code>uint32_t</code> operation counter within that log record. When a log record reflects something log record that is not a transaction, such as the start of a checkpoint, the operation counter returned for the key will be zero. When the log record maps to a transaction, the first log record returned, with an operation counter of zero, will be the entire log record. Then the <a class="el" href="struct_w_t___c_u_r_s_o_r.html#a0503f16bd8f3d05aa3552f229b3a8e1b" title="Return the next record.">WT_CURSOR::next</a> call will step into the transaction and return the first individual operation within that transaction and each additional individual operation, adding one to the operation counter for each one. A transaction with a single operation will return two records related to that transaction.</p>

<p>The unique log record identifier maps to a WT_LSN data structure, which has two fields: WT_LSN::file, the log file identifier, and WT_LSN::offset, the offset of the log record in the log file.</p>

<p>Here is an example of getting the log cursor's key:</p>

<div class="fragment"><div class="line"> error_check(cursor-><a class="code" href="struct_w_t___c_u_r_s_o_r.html#af19f6f9d9c7fc248ab38879032620b2f">get_key</a>(cursor, &log_file, &log_offset, &opcount));</div>

</div><p> The log cursor's value is composed of six fields:</p>

<ul>

<li>a <code>uint64_t</code> transaction ID (set for commit records only, otherwise 0),</li>

<li>a <code>uint32_t</code> log record type</li>

<li>a <code>uint32_t</code> operation type (set for commit records only, otherwise 0)</li>

<li>a <code>uint32_t</code> file id (if applicable, otherwise 0)</li>

<li>the operation key (commit records only, otherwise empty)</li>

<li>the operation value</li>

</ul>

<p>The transaction ID may not be unique across recovery, that is, closing and reopening the database may result in transaction IDs smaller than previously seen transaction IDs.</p>

<p>The log record and log operation types are taken from <a class="el" href="group__wt.html#log_types">log_types</a>; typically, the only record or operation type applications are concerned with is <a class="el" href="group__wt.html#ga8cf04a003979e7852209079d4093d783" title="Message.">WT_LOGREC_MESSAGE</a>, which is a log record generated by the application.</p>

<p>The file ID may not be unique across recovery, that is, closing and reopening the database may result in file IDs changing. Additionally, there is currently no way to map file IDs to file names or higher-level objects.</p>

<p>Here is an example of getting the log cursor's value:</p>

<div class="fragment"><div class="line"> error_check(cursor-><a class="code" href="struct_w_t___c_u_r_s_o_r.html#af85364a5af50b95bbc46c82e72f75c01">get_value</a>(</div>

<div class="line"> cursor, &txnid, &rectype, &optype, &fileid, &logrec_key, &logrec_value));</div>

</div><p> For clarity, imagine a set of three log records:</p>

<ul>

<li>the first recording an internal operation, say a checkpoint start,</li>

<li>the second committing a transaction with three operations,</li>

<li>the third committing a transaction with a single operation.</li>

</ul>

<p>The log cursor's <a class="el" href="struct_w_t___c_u_r_s_o_r.html#a0503f16bd8f3d05aa3552f229b3a8e1b" title="Return the next record.">WT_CURSOR::next</a> call will return a total of seven records. Here's an example of what it would look like:</p>

<li>LSN=[1,1000], operation counter=0, transaction ID=0, value of checkpoint start</li>

<li>LSN=[1,2000], operation counter=0, transaction ID=30, all operations of transaction 30</li>

<li>LSN=[1,2000], operation counter=1, transaction ID=30, first operation of transaction 30</li>

<li>LSN=[1,2000], operation counter=2, transaction ID=30, second operation of transaction 30</li>

<li>LSN=[1,2000], operation counter=3, transaction ID=30, third operation of transaction 30</li>

<li>LSN=[1,3000], operation counter=0, transaction ID=31, all operations of transaction 31</li>

<li>LSN=[1,3000], operation counter=1, transaction ID=31, only operation of transaction 31</li>

</ol>

<p>The first time the log cursor will return a key with a unique log ID, no transaction ID, and an operation counter of 0. The next six returns from the log cursor will have a common log ID, a common transaction ID, and operation counters starting at 0, returning the whole record and then starting at 1 and ending at 5 for each of the five individual operations. The next return from the log cursor will again have a unique log ID, a unique transaction ID, and an operation counter of 0. And the final return from the log cursor will have an operation counter of 1.</p>

<p>Here's a more complete example of walking the log and displaying the results:</p>

<div class="fragment"><div class="line"><span class="keyword">static</span> <span class="keywordtype">void</span></div>

<div class="line">print_record(uint32_t log_file, uint32_t log_offset, uint32_t opcount, uint32_t rectype,</div>

<div class="line"> uint32_t optype, uint64_t txnid, uint32_t fileid, <a class="code" href="group__wt.html#struct_w_t___i_t_e_m">WT_ITEM</a> *key, <a class="code" href="group__wt.html#struct_w_t___i_t_e_m">WT_ITEM</a> *value)</div>

<div class="line"> printf(<span class="stringliteral">"LSN [%"</span> PRIu32 <span class="stringliteral">"][%"</span> PRIu32 <span class="stringliteral">"].%"</span> PRIu32 <span class="stringliteral">": record type %"</span> PRIu32 <span class="stringliteral">" optype %"</span> PRIu32</div>

<div class="line"> <span class="stringliteral">" txnid %"</span> PRIu64 <span class="stringliteral">" fileid %"</span> PRIu32,</div>

<div class="line"> log_file, log_offset, opcount, rectype, optype, txnid, fileid);</div>

<div class="line"> printf(<span class="stringliteral">" key size %zu value size %zu\n"</span>, key-><a class="code" href="group__wt.html#abd8b1299dfb5c54dc049fa42a4680b55">size</a>, value-><a class="code" href="group__wt.html#abd8b1299dfb5c54dc049fa42a4680b55">size</a>);</div>

<div class="line"> <span class="keywordflow">if</span> (rectype == <a class="code" href="group__wt.html#ga8cf04a003979e7852209079d4093d783">WT_LOGREC_MESSAGE</a>)</div>

<div class="line"> printf(<span class="stringliteral">"Application Record: %s\n"</span>, (<span class="keywordtype">char</span> *)value-><a class="code" href="group__wt.html#a57f5e62aa968275d7e398cfa70e49450">data</a>);</div>

<div class="line"><span class="comment"> * simple_walk_log --</span></div>

<div class="line"><span class="comment"> * A simple walk of the log.</span></div>

<div class="line"><span class="keyword">static</span> <span class="keywordtype">void</span></div>

<div class="line">simple_walk_log(<a class="code" href="struct_w_t___s_e_s_s_i_o_n.html">WT_SESSION</a> *session, <span class="keywordtype">int</span> count_min)</div>

<div class="line"> <a class="code" href="struct_w_t___c_u_r_s_o_r.html">WT_CURSOR</a> *cursor;</div>

<div class="line"> <a class="code" href="group__wt.html#struct_w_t___i_t_e_m">WT_ITEM</a> logrec_key, logrec_value;</div>

<div class="line"> uint64_t txnid;</div>

<div class="line"> uint32_t fileid, log_file, log_offset, opcount, optype, rectype;</div>

<div class="line"> <span class="keywordtype">int</span> count, ret;</div>

<div class="line"> error_check(session-><a class="code" href="struct_w_t___s_e_s_s_i_o_n.html#afb5b4a69c2c5cafe411b2b04fdc1c75d">open_cursor</a>(session, <span class="stringliteral">"log:"</span>, NULL, NULL, &cursor));</div>

<div class="line"> count = 0;</div>

<div class="line"> <span class="keywordflow">while</span> ((ret = cursor-><a class="code" href="struct_w_t___c_u_r_s_o_r.html#a0503f16bd8f3d05aa3552f229b3a8e1b">next</a>(cursor)) == 0) {</div>

<div class="line"> count++;</div>

<div class="line"> error_check(cursor-><a class="code" href="struct_w_t___c_u_r_s_o_r.html#af19f6f9d9c7fc248ab38879032620b2f">get_key</a>(cursor, &log_file, &log_offset, &opcount));</div>

<div class="line"> error_check(cursor-><a class="code" href="struct_w_t___c_u_r_s_o_r.html#af85364a5af50b95bbc46c82e72f75c01">get_value</a>(</div>

<div class="line"> cursor, &txnid, &rectype, &optype, &fileid, &logrec_key, &logrec_value));</div>

<div class="line"> print_record(log_file, log_offset, opcount, rectype, optype, txnid, fileid, &logrec_key,</div>

<div class="line"> &logrec_value);</div>

<div class="line"> scan_end_check(ret == <a class="code" href="group__wt.html#ga3c9e1b494d95cf34404ab7a974af6bf8">WT_NOTFOUND</a>);</div>

<div class="line"> error_check(cursor-><a class="code" href="struct_w_t___c_u_r_s_o_r.html#aeea071f192cab12245a50fbe71c3460b">close</a>(cursor));</div>

<div class="line"> <span class="keywordflow">if</span> (count < count_min) {</div>

<div class="line"> fprintf(stderr, <span class="stringliteral">"Expected minimum %d records, found %d\n"</span>, count_min, count);</div>

</div><p> The log cursor's key can be used to search for specific records in the log (assuming the record still exists and has not been removed), by setting the key and calling <a class="el" href="struct_w_t___c_u_r_s_o_r.html#a7e25b2ced2cf3ec68bd5429bf921c79f" title="Return the record matching the key.">WT_CURSOR::search</a>. However, it is not possible to search for a specific operation within a log record, and the key's operation counter is ignored when the key is set. The result of a search for a log record with more than one operation is always the first operation in the log record.</p>

<p>Here is an example of setting the log cursor's key:</p>

<div class="fragment"><div class="line"> cursor-><a class="code" href="struct_w_t___c_u_r_s_o_r.html#ad1088d719df40babc1f57d086691ebdc">set_key</a>(cursor, save_file, save_offset, 0);</div>

</div><p> Log cursors are read-only, however applications can insert their own log records using <a class="el" href="struct_w_t___s_e_s_s_i_o_n.html#ac85541ca42d7596857d9f50e03c78eb5" title="Insert a WT_LOGREC_MESSAGE type record in the database log files (the database must be configured for...">WT_SESSION::log_printf</a>. Here is an example of adding an application record into the database log:</p>

<div class="fragment"><div class="line"> error_check(session-><a class="code" href="struct_w_t___s_e_s_s_i_o_n.html#ac85541ca42d7596857d9f50e03c78eb5">log_printf</a>(session, <span class="stringliteral">"Wrote %d records"</span>, record_count));</div>

</div></div></div>

</div>

</div>

<div class="ttc" id="astruct_w_t___s_e_s_s_i_o_n_html_afb5b4a69c2c5cafe411b2b04fdc1c75d"><div class="ttname"><a href="struct_w_t___s_e_s_s_i_o_n.html#afb5b4a69c2c5cafe411b2b04fdc1c75d">WT_SESSION::open_cursor</a></div><div class="ttdeci">int open_cursor(WT_SESSION *session, const char *uri, WT_CURSOR *to_dup, const char *config, WT_CURSOR **cursorp)</div><div class="ttdoc">Open a new cursor on a data source or duplicate an existing cursor.</div></div>

<div class="ttc" id="agroup__wt_html_a57f5e62aa968275d7e398cfa70e49450"><div class="ttname"><a href="group__wt.html#a57f5e62aa968275d7e398cfa70e49450">WT_ITEM::data</a></div><div class="ttdeci">const void * data</div><div class="ttdoc">The memory reference of the data item.</div><div class="ttdef"><b>Definition:</b> wiredtiger.h.in:113</div></div>

<div class="ttc" id="astruct_w_t___c_u_r_s_o_r_html_af19f6f9d9c7fc248ab38879032620b2f"><div class="ttname"><a href="struct_w_t___c_u_r_s_o_r.html#af19f6f9d9c7fc248ab38879032620b2f">WT_CURSOR::get_key</a></div><div class="ttdeci">int get_key(WT_CURSOR *cursor,...)</div><div class="ttdoc">Get the key for the current record.</div></div>

<div class="ttc" id="astruct_w_t___c_u_r_s_o_r_html"><div class="ttname"><a href="struct_w_t___c_u_r_s_o_r.html">WT_CURSOR</a></div><div class="ttdoc">A WT_CURSOR handle is the interface to a cursor.</div><div class="ttdef"><b>Definition:</b> wiredtiger.h.in:212</div></div>

<div class="ttc" id="agroup__wt_html_abd8b1299dfb5c54dc049fa42a4680b55"><div class="ttname"><a href="group__wt.html#abd8b1299dfb5c54dc049fa42a4680b55">WT_ITEM::size</a></div><div class="ttdeci">size_t size</div><div class="ttdoc">The number of bytes in the data item.</div><div class="ttdef"><b>Definition:</b> wiredtiger.h.in:122</div></div>

<div class="ttc" id="astruct_w_t___c_u_r_s_o_r_html_a0503f16bd8f3d05aa3552f229b3a8e1b"><div class="ttname"><a href="struct_w_t___c_u_r_s_o_r.html#a0503f16bd8f3d05aa3552f229b3a8e1b">WT_CURSOR::next</a></div><div class="ttdeci">int next(WT_CURSOR *cursor)</div><div class="ttdoc">Return the next record.</div></div>

<div class="ttc" id="agroup__wt_html_struct_w_t___i_t_e_m"><div class="ttname"><a href="group__wt.html#struct_w_t___i_t_e_m">WT_ITEM</a></div><div class="ttdoc">A raw item of data to be managed, including a pointer to the data and a length.</div><div class="ttdef"><b>Definition:</b> wiredtiger.h.in:105</div></div>

<div class="ttc" id="astruct_w_t___c_u_r_s_o_r_html_af85364a5af50b95bbc46c82e72f75c01"><div class="ttname"><a href="struct_w_t___c_u_r_s_o_r.html#af85364a5af50b95bbc46c82e72f75c01">WT_CURSOR::get_value</a></div><div class="ttdeci">int get_value(WT_CURSOR *cursor,...)</div><div class="ttdoc">Get the value for the current record.</div></div>

<div class="ttc" id="astruct_w_t___c_u_r_s_o_r_html_aeea071f192cab12245a50fbe71c3460b"><div class="ttname"><a href="struct_w_t___c_u_r_s_o_r.html#aeea071f192cab12245a50fbe71c3460b">WT_CURSOR::close</a></div><div class="ttdeci">int close(WT_CURSOR *cursor)</div><div class="ttdoc">Close the cursor.</div></div>

<div class="ttc" id="astruct_w_t___s_e_s_s_i_o_n_html_ac85541ca42d7596857d9f50e03c78eb5"><div class="ttname"><a href="struct_w_t___s_e_s_s_i_o_n.html#ac85541ca42d7596857d9f50e03c78eb5">WT_SESSION::log_printf</a></div><div class="ttdeci">int log_printf(WT_SESSION *session, const char *format,...)</div><div class="ttdoc">Insert a WT_LOGREC_MESSAGE type record in the database log files (the database must be configured for...</div></div>

<div class="ttc" id="astruct_w_t___c_u_r_s_o_r_html_ad1088d719df40babc1f57d086691ebdc"><div class="ttname"><a href="struct_w_t___c_u_r_s_o_r.html#ad1088d719df40babc1f57d086691ebdc">WT_CURSOR::set_key</a></div><div class="ttdeci">void set_key(WT_CURSOR *cursor,...)</div><div class="ttdoc">Set the key for the next operation.</div></div>

<div class="ttc" id="agroup__wt_html_ga3c9e1b494d95cf34404ab7a974af6bf8"><div class="ttname"><a href="group__wt.html#ga3c9e1b494d95cf34404ab7a974af6bf8">WT_NOTFOUND</a></div><div class="ttdeci">#define WT_NOTFOUND</div><div class="ttdoc">Item not found.</div><div class="ttdef"><b>Definition:</b> wiredtiger.h.in:4240</div></div>

<div class="ttc" id="astruct_w_t___s_e_s_s_i_o_n_html"><div class="ttname"><a href="struct_w_t___s_e_s_s_i_o_n.html">WT_SESSION</a></div><div class="ttdoc">All data operations are performed in the context of a WT_SESSION.</div><div class="ttdef"><b>Definition:</b> wiredtiger.h.in:827</div></div>

<div class="ttc" id="agroup__wt_html_ga8cf04a003979e7852209079d4093d783"><div class="ttname"><a href="group__wt.html#ga8cf04a003979e7852209079d4093d783">WT_LOGREC_MESSAGE</a></div><div class="ttdeci">#define WT_LOGREC_MESSAGE</div><div class="ttdoc">Message.</div><div class="ttdef"><b>Definition:</b> wiredtiger.h.in:6057</div></div>

<ul>

<li class="navelem"><a class="el" href="index.html">Reference Guide</a></li><li class="navelem"><a class="el" href="programming.html">Writing WiredTiger applications</a></li>

</ul>

</div>

</body>

</html>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FilesExpand file tree

cursor_log.html

Latest commit

History

cursor_log.html

File metadata and controls