rt-serializer - Man Page
Serialize an RT database to disk
Synopsis
rt-validator --check && rt-serializer
This script is used to write out the entire RT database to disk, for later import into a different RT instance. It requires that the data in the database be self-consistent, in order to do so; please make sure that the database being exported passes validation by rt-validator before attempting to use rt-serializer.
While running, it will attempt to estimate the number of remaining objects to be serialized; these estimates are pessimistic, and will be incorrect if --no-users, --no-groups, or --no-tickets are used.
If the controlling terminal is large enough (more than 25 columns high) and the gnuplot program is installed, it will also show a textual graph of the queue size over time.
Options
- --directory name
The name of the output directory to write data files to, which should not exist yet; it is a fatal error if it does. Defaults to
./$Organization:Date/, where$Organizationis as set in RT_SiteConfig.pm, and Date is today's date.- --force
Remove the output directory before starting.
- --size megabytes
By default,
rt-serializerchunks its output into data files which are around 32Mb in size; this option is used to set a different threshold size, in megabytes. Note that this is the threshold after which it rotates to writing a new file, and is as such the lower bound on the size of each output file.- --no-users
By default, all privileged users are serialized; passing
--no-userslimits it to only those users which are referenced by serialized tickets and history, and are thus necessary for internal consistency.- --no-groups
By default, all groups are serialized; passing
--no-groupslimits it to only system-internal groups, which are needed for internal consistency.- --no-assets
By default, all assets are serialized; passing
--no-assetsskips assets during serialization.- --no-disabled
By default, all queues, custom fields, etc, including disabled ones, are serialized; passing
--no-disabledskips such disabled records during serialization.- --no-deleted
By default, all tickets and assets, including deleted ones, are serialized; passing
--no-deletedskips deleted tickets and assets during serialization.- --scrips
No scrips or templates are serialized by default; this option forces all scrips and templates to be serialized.
- --acls
No ACLs are serialized by default; this option forces all ACLs to be serialized.
- --no-tickets
Skip serialization of all ticket data.
- --limit-queues
Takes a list of queue IDs or names separated by commas. When provided, only that set of queues (and the tickets in them) will be serialized.
- --limit-cfs
Takes a list of custom field IDs or names separated by commas. When provided, only that set of custom fields will be serialized.
- --hyperlink-unmigrated
Replace links to local records which are not being migrated with hyperlinks. The hyperlinks will use the serializing RT's configured URL.
Without this option, such links are instead dropped, and transactions which had updated such links will be replaced with an explanatory message.
- --no-transactions
Skip serialization of all transactions on any records (not just tickets).
- --clone
Serializes your entire database, creating a clone. This option should be used if you want to migrate your RT database from one database type to another (e.g. MySQL to PostgreSQL). It is an error to combine
--clonewith any option that limits object types serialized. No dependency walking is performed when cloning.rt-importerwill detect that your serialized data set was generated by a clone.- --all
Serializes your entire database, creating a clone-like data. Both
--alland--clonedo not check dependencies, the difference is--allgenerates UIDs: it means the ids in source instance do not necessarily be synced to target instance, which makes it quite useful to fully merge multiple RT instances. Use--cloneinstead if you really want to keep ids in source instance.- --incremental
Will generate an incremental serialized dataset using the data stored in your IncrementalRecords database table. This assumes that you have created that table and run RT using the Record_Local.pm shim as documented in
docs/incremental-export/.- --gc n
Adjust how often the garbage collection sweep is done; lower numbers are more frequent. See "Garbage Collection".
- --page n
Adjust how many rows are pulled from the database in a single query. Disable paging by setting this to 0. Defaults to 100.
Keep in mind that rows from RT's Attachments table are the limiting factor when determining page size. You should likely be aiming for 60-75% of your total memory on an otherwise unloaded box.
- --quiet
Do not show graphical progress UI.
- --verbose
Do not show graphical progress UI, but rather log was each row is written out.
Garbage Collection
rt-serializer maintains a priority queue of objects to serialize, or searches which may result in objects to serialize. When inserting into this queue, it does no checking if the object in question is already in the queue, or if the search will contain any results. These checks are done when the object reaches the front of the queue, or during periodic garbage collection.
During periodic garbage collection, the entire queue is swept for objects which have already been serialized, occur more than once in the queue, and searches which contain no results in the database. This is done to reduce the memory footprint of the serialization process, and is triggered when enough new objects have been placed in the queue. This parameter is tunable via the --gc parameter, which defaults to running garbage collection every 5,000 objects inserted into the queue; smaller numbers will result in more frequent garbage collection.
The default of 5,000 is roughly tuned based on a database with several thousand tickets, but optimal values will vary wildly depending on database configuration and size. Values as low as 25 have provided speedups with smaller databases; if speed is a factor, experimenting with different --gc values may be helpful. Note that there are significant boundary condition changes in serialization rate, as the queue empties and fills, causing the time estimates to be rather imprecise near the start and end of the process.
Setting --gc to 0 turns off all garbage collection. Be aware that this will bloat the memory usage of the serializer. Any negative value for --gc turns off periodic garbage collection and instead objects already serialized or in the queue are checked for at the time they would be inserted.