Registry Compliance API (RELEASE) Version 1.1.2-Release |
Copyright © 2003-2006 Unspam Registry Services, Inc. All rights reserved. |
This document explains in detail the semantics of the function calls you can make using the Registry Compliance API service. This service is limited to access by approved senders and E-mail Service Providers (ESPs) who have (1) completed the information required by the governments to access their compliance programs, (2) been approved by these governments for access, and (3) agreed to the required license terms.
In this document, you will learn:
This document is subject to change. The latest version of this document is always maintained at:
https://www.registrycompliance.com/api_spec.html
We have built prototypes of applications to interact with the Registry Compliance system. To see these prototypes, please visit:
https://www.registrycompliance.com/samples/
If you have comments, find errors, or have questions, please contact apisupport@registrycompliance.com.
Interaction with the Registry Compliance system is accomplished with POST requests through the secure HTTP protocol (HTTPS). This protocol was chosen for its simplicity, network administrators' familiarity with it, its ability to pass through many corporate firewalls without requiring their modification, the protocol's extensive documentation, and the wide availability of access tools written in many languages.
Some links concerning establishing and performing POST requests over secure HTTP sessions are provided below:
There are some general issues that you should keep in mind when designing an application for interacting with this API. You may want to refer to this checklist as you are constructing the application. Neglecting to follow these general guidelines is the usual source of difficulty for application designers.
The Registry Compliance system was designed to maintain the privacy and security of the lists that are uploaded for scrubbing. At no time do the e-mail addresses or other contact points being scrubbed leave the sender's client machine. Instead, only hashes of the e-mail addresses or other contact points are transmitted. To ensure this level of security, a sender's list must be hashed and formatted before being uploaded to the Registry Compliance system.
Entries on a sender's list should be hashed using MD5 hashing. MD5 was chosen for its speed and wide availability across multiple programming languages (see RFC 1321).
Binary and HEX MD5 is supported natively or in widely available libraries across most modern languages. References to some of these libraries are listed below:
Before hashing, each entry must be combined with the entry's type and the current salt provided by the Registry Compliance system. An entry's type is pre-pended to the beginning entry; salts are appended to the end of each entry. The MD5 function should generally be formed as follows:
Types are represented by a three-character, upper-case code. Currently, the system supports all of the following types:
Please ensure that type codes are always rendered in uppercase when constructing your MD5 hashing function. The actual entry, on the other hand, should be reduced to lower case before hashing. In other words:
Do not remove valid characters from e-mail addresses.
For types FAX and SMS, any non-numeric characters (including spaces) should be stripped before hashing. Area codes must be included in phone numbers. Any leading "1" should be stripped from the number. A 10-digit number is expected. For example:
For types TWT, IGR, SNP, YTB, TOK, TCH, and DCR the prefixed at-sign ("@") should be stripped if if exists. For example:
The salt is appended to the end of the entry. Section 3 in this document provides information on how to interact with the Registry Compliance system in order to obtain the most recent version of salt. Salts are at least 128 characters long, and may contain up to 180 characters. Within this range, however, do not rely on the salt's length to remain constant over time. Salts may contain non-alphanumeric characters, including any ASCII symbol. Salts are passed from the Registry Compliance systems in a URL-encoded format (see RFC 1738) and should be URL-decoded before use. Detailed instructions for receiving and decoding salts appear in Section 3 of this document.
Only the entry's case should be lowered. Ensure you do not strip characters or change the case of the pre-pended type or appended salt before hashing. There is no delimiter between the type, the entry, and the salt. The salt will vary in length and will rotate at a random interval over time.
Please note: two salts are returned by the Registry Compliance system ("salta" and "saltb"). When formatting your list for upload to the system, you should append "salta" to the contact points you are uploading. The second salt ("saltb") is used when results are returned.
Two examples are provided below.
EXAMPLE 1 |
|
Type: | E-Mail Address |
Entry: | John.Doe@example.com |
Salt: |
LKJAWlkj234ljAKsjdlk34098()Aipioe3j+Aw4e5rawerWE@:#4wer;wer:#4q23$:awerawseR:W34;AQW3RE:AWER;:234!@#;wer:$#5AWe;afvzsdf';ws';re[]
|
MD5 Input String: |
EMLjohn.doe@example.comLKJAWlkj234ljAKsjdlk34098()Aipioe3j+Aw4e5rawerWE@:#4wer;wer:#4q23$:awerawseR:W34;AQW3RE:AWER;:234!@#;wer:$#5AWe;afvzsdf';ws';re[]
|
EXAMPLE 2 |
|
Type: | Yahoo Instant Messenger Id |
Entry: | ExAMpLE |
Salt: |
lkwjerlj3lASLDKJklakjdaISJDlakjAWLEJlkjdalkSJELAWKjelMASldkjalskdjL'AKSDMKjlkJAsdjlaalkwjladkjalskjdlaksjdlAkmsdlkjalw][234pa)(*3opilkjLASDjkn23
|
MD5 Input String: |
YIMexamplelkwjerlj3lASLDKJklakjdaISJDlakjAWLEJlkjdalkSJELAWKjelMASldkjalskdjL'AKSDMKjlkJAsdjlaalkwjladkjalskjdlaksjdlAkmsdlkjalw][234pa)(*3opilkjLASDjkn23
|
If you choose to transmit binary data, you may wish to hash each entry into HEX for storage in your system, but convert it to binary during transmission. Alternately, you may generate both binary and HEX versions of each entry, retaining the HEX version until you receive the returned results. In either case, you should map each HEX MD5 hash to its associated entry in order to do a reverse comparison when results are returned.
Your MD5 function should output a 16-byte binary result, regardless of the length of the contact point or the appended current salt value. Binary hash output should then be appended to a file. It is critical that this file be in binary format. Typically you can configure your MD5 function to generate binary rather than HEX. Alternatively, you may output HEX and then convert the file to binary. Converting from HEX may be preferable since you will need a HEX version of the MD5 hash for your final results comparison, described in Section 4 of this document.
If you do a conversion from HEX to binary there is a special consideration on some Windows-based machines. If your MD5 function generates uppercase HEX, you should convert those characters to lowercase before the binary conversion. We have found this resolves a number of problems when converting HEX. Always when converting from a non-binary format, ensure that the conversion is done byte-wise with the high-nibble first.
A file of hashed entries should be prepared for transmission to the Registry Compliance system. If you are checking multiple types of entries, each file must contain only one type. In other words, you may not mix hashes of e-mail addresses with hashes of Yahoo Instant Messenger Ids.
The hashed entries file should be stored in HEX format, but may be transmitted to the system in either binary or HEX. Binary transmission, however, is preferred, since binary data is half the size of HEX data. When transmitting entries to the system, they should be simply appended, one after the other, with no delimiter. Since all hashed entries are a common number of bytes (16 for binary or 32 for HEX), the individual entries can be subdivided, even without a delimeter, once the file is uploaded. No carriage returns, line breaks, or other extraneous characters should be inserted into the file. Binary files should be byte-wise with the high-nibble first. The actual name of the file does not matter.
Each file can contain a maximum of 2.5 million entries, or approximately 40 megabytes. Attempting to upload a single file exceeding 40 megabytes will trigger an abuse event, which may restrict your access to the system. If the list that you are checking exceeds 2.5 million entries, you may upload multiple files under a single "check task" job. However, the files should be split in such a way that the file size is divisible by 16 bytes for binary files or 32 bytes for HEX files. In other words, you should not split a hashed entry into more than one file. Section 3 in this document describes the process of associating multiple files with a single task for lists exceeding 2.5 million entries.
In order to accommodate domain-wide registrations on the Children's Protection Registries, certain considerations are in place for e-mail addresses. Because it is impossible to know the original e-mail addresses from the uploaded hash file, and therefore impossible to determine the domains associated with those e-mail addresses, a separate file of hashed domains must also be uploaded and associated under the same task.
Domains should be converted to lower case before they are hashed. Each domain should be included on the domain list only once, even if it is shared by multiple e-mail addresses. Based on current government regulations, when a list of domains is run in conjunction with a list of e-mail addresses, there is no per-check charge for checking the associated domains.
Domains should be hashed using the type DMN. See the example below:
EXAMPLE 3 | |
Original E-Mail Address List: | johndoe@example.com janesmith@example.com littlesusy@anotherexample.com |
Associated Domains: | example.com anotherexample.com |
Salt: | lkwjerlj3lASLDKJklakjdaISJDlakjAWLEJlkjdalkSJELAWKjelMASldkjalskdjL'AKSDMKjlkJAsdjlaalkwjladkjalskjdlaksjdlAkmsdlkjalw][234pa)(*3opilkjLASDjkn23 |
First MD5 Input String: | DMNexample.comlkwjerlj3lASLDKJklakjdaISJDlakjAWLEJlkjdalkSJELAWKjelMASldkjalskdjL'AKSDMKjlkJAsdjlaalkwjladkjalskjdlaksjdlAkmsdlkjalw][234pa)(*3opilkjLASDjkn23 |
Second MD5 Input String: | DMNanotherexample.comlkwjerlj3lASLDKJklakjdaISJDlakjAWLEJlkjdalkSJELAWKjelMASldkjalskdjL'AKSDMKjlkJAsdjlaalkwjladkjalskjdlaksjdlAkmsdlkjalw][234pa)(*3opilkjLASDjkn23 |
In order to ensure that the file is formatted correctly and no errors are introduced during transmission, we implement two checking systems: a verification entry and a file checksum.
The first entry in each file must be a special verification hash entry. This verification hash entry varies for each file type. The entries to create this verification hash are below:
You should hash each of the entries above just as you would hash any other entry on your list. Since the intent of this check is to ensure that your hashing function is working correctly, you should run the entries above through the same function as you do for any other entry on your list. The verification entries above are intentionally formatted in such a way as to test that you are properly reformatting them (for example, they include uppercase letters).
If you upload multiple files as part of a single task, you must include a verification entry at the beginning of each file. This is true even if you attach multiple files containing the same type of entry within a single task.
In addition to the verification check, you must generate a checksum of the entire file. To do that, you should use MD5 to hash the entire file. You should either setup the MD5 function to output HEX, or convert the binary output to HEX. This HEX hash code functions as a checksum and you include it with the file during upload. Instructions on how this checksum should be passed with your upload request are discussed at length in Section 3 of this document. The Registry Compliance system generates its own MD5 checksum of the file upon receipt. If these two checksums do not match, then an error condition is returned. This ensures that the file you upload is not corrupted during transmission.
Every operation request from a client application to the Registry Compliance system is accomplished as a POST request through the HTTPS protocol. For more information on passing form data via POST requests, including transferring files, see RFC 1867. Below is an example of a complete POST request for the "TASK_ADD" operation (defined in section 3.2.3). This operation is one of the more complicated tasks an application will need to initiate with the server. Generally, all POST requests will follow this basic format with modifications for their particular requirements. Libraries that handle POST requests, such as LWP in Perl, will automatically generate such a request with the data you provide.
Some libraries allow you to specify the multi-part boundary, while others will set this boundary for you. This "app-defined boundary text" can be any text which delimits the boundary between parts in the POST request. Because for some requests you will be passing a substantial portion of hashed data, which is generally random, you should choose a boundary which is relatively long (at least 50 characters) and unlikely to appear at random within even several megabytes worth of hashes. For more information on choosing a boundary, see RFC 1867.
We have included simple code samples, written in common languages, to assist you in developing your POST functionality. Please click the link below that corresponds to your development platform:
All operations return an XML response to indicate whether the POST was completed successfully. They furthermore include a timestamp for the operation and a transaction code to uniquely identify the particular operation. Most tasks also return additional data such as a task ID, the status of a current task, etc. At a minimum, successful server requests will return at least the following:
In the REQUEST portion of the XML response, the "OP" code is echoed back in order to help you keep track of which operation obtained the returned result. If a request has an associated "task key", then that information is also echoed as part of the REQUEST portion of the XML response. While the basic hierarchical structure of each XML response will be as they appear in the examples in this document, you should not rely on the order of the XML results to be identical to the examples that appear here.
Within the RESPONSE portion of the XML, the TIMESTAMP indicates the local Registry Compliance server time in RFC 822 format. The TRANSACTION field uniquely identifies the particular transaction with a 35-character string and should be reported to the Registry Compliance Customer Support Staff if a particular transaction needs to be audited. The ISTEST field will only be present if you are querying against one of the test servers. If operations are submitted to these servers, information regarding the connection to that server will be contained in this field. Additional information will also be returned for successful requests, depending on the operation. These additional XML fields are described under the various "OP" codes described in Section 3.2 of this document.
Unsuccessful requests will return the following:
Specific ERRCODEs are always 3 digit numbers. ERRMSGs contain an explanation of the error that was encountered and what can be done to remedy it. For an explanation of the specific ERRCODEs and ERRMSGs, see Section 5 of this document.
Every POST request must include at least the following three basic parameters:
Senders and ESPs who have been approved for access to the API are issued an API key. API keys are 96 characters in length and are issued only to senders and ESPs who have been approved for access to the API. API keys are also associated with a particular set of IP addresses. Requests using a particular API key may only be initiated from one of the associated IP addresses. API-approved senders and ESPs may list the IP addresses they would like to access the API from when applying for access to the API.
In addition to the API key, every request must be associated with a particular account key. To receive an account key, governments require each individual or company accessing the API to submit certain basic information. For more information, see the Registry Compliance website.
To define which request is being made, the client should POST an "OP" code. "OP" codes are used to retrieve the current salts, begin a new task, add files to a task, begin the scrubbing process on a task, retrieve the results from a task, or cancel a task that has been started. Depending on the particular request, additional parameters may also be required. Specific "OP" codes are described in Section 3.2 below.
Because certain operations must be multipart form data posts, under the specifications of RFC 1867 each field in the POST request must be listed in a separate part. This can be tedious to code if you are using certain less-friendly libraries to accomplish your POST request or, especially, if you are using raw sockets. To provide a shortcut, we allow all the required request fields to be passed in a single, optional parameter: "request_params".
When using "request_params" to submit multiple variables via a single parameter, each variable should be formatted as if it were being posted through a URL GET request (see RFC 1738). In other words, it should consist of the variable name, followed by an equals sign, followed by the value of the variable. Multiple variable/value pairs should be separated by ampersands. All data should be URL-encoded to avoid potential transmission errors (see RFC 1738). The order of the individual variables in the "request_params" field does not matter. See the following example for the "TASK_ADD" operation:
Any file data must still be passed in a second part (see, for example, the "TASK_ADD" operation described below). However, using this shortcut limits the total number of parts of a POST request to a maximum of two. You do not have to use the "request_params" field if the library you are using makes it easy to post each field required for a task individually.
If you elect to submit multiple parameters, rather than using "request_params", the order of the parameters included in the POST does not matter. You should not submit "request_params" if you submit individual parameters. If you do so, "request_params" will take priority. Please use one method or the other to submit parameters. To avoid confusion, do not mix both in a single operation.
Interaction with the Registry Compliance system is accomplished through the use of various operations. These operations are commenced with predefined "OP" codes. These "OP" codes are passed via the OP parameter as part of the POST request. The currently supported "OP" codes are listed in the following subsections, along with descriptions of each code's purpose.
This "OP" code is sent in order to get the current salts used for hashing addresses. It does not need to be run before every transaction, but should be run if the "TASK_START" operation returns a result that the client application is currently using expired salts. This allows an API user to store a hashed version of a sender's list without having to rehash existing entries until the salt rotates.
Input: |
Requires only basic parameters described in Section 3.1 of this document. Unlike most other "OP" codes, the "GET_SALTS" code does not require a task key. |
Output: |
If successful, returns the basic response plus SALTA and SALTB. SALTA and SALTB are between 128 and 180 characters in length. They are URL-encoded (see RFC 1738) and must be URL-decoded before being used. When decoded, salts may contain any ASCII character.
<XML>
<REQUEST>
<OP>GET_SALTS</OP>
</REQUEST>
<RESPONSE>
<TIMESTAMP>[formatted like Sun, 17 Jul 2005 01:51:35 -0400]</TIMESTAMP>
<RESULT>SUCCESS</RESULT>
<TRANSACTION>[35 characters]</TRANSACTION>
<SALTA>[128-180 URL-encoded characters]</SALTA>
<SALTB>[128-180 URL-encoded characters]</SALTB>
<ISTEST>[If present, you are connected to a test server. This field will contain information about that connection]</ISTEST>
</RESPONSE>
</XML>
SALTA is used for the initial hashing of the entries on a marketer's list. The process by which SALTA is used is described in Section 2 of this document. SALTB is used to create an entry signature to finalize the entries to be scrubbed. The process by which SALTB is used is described in Section 4 of this document. |
This "OP" code is sent to begin a new scrubbing task. Each task may contain more than one file, and while each file must contain only a single type of entry, each task can scrub multiple files of different types at once. Once a new task is started, you will receive a task key. Most other "OP" codes, such as those used to upload files or cancel a task, require this task key. It should be retained by the client application at least until the task is complete.
Input: | Requires the basic parameters described in Section 3.1 of this document. In addition, to begin a new task you must pass the following parameters via the POST request.
|
Output: | If the information to start a task is valid and no errors occur, the "TASK_START" "OP" code will return a task key. The task key should be stored by the client application at least until a task is completed as it will be needed for most task-related operations. A successful response will look like this:
<XML>
<REQUEST>
<OP>TASK_START</OP>
<TASK_KEY>[32 characters]</TASK_KEY>
</REQUEST>
<RESPONSE>
<TIMESTAMP>[formatted like Sun, 17 Jul 2005 01:51:35 -0400]</TIMESTAMP>
<RESULT>SUCCESS</RESULT>
<TASK_KEY>[32 characters]</TASK_KEY>
<TASK_ADD_DOMAIN>[domain to which all subsequent "TASK_ADD" operations should be posted]</TASK_ADD_DOMAIN>
<ISTEST>[If present, you are connected to a test server. This field will contain information about that connection]</ISTEST>
</RESPONSE>
</XML>
There are two probable reasons that a "TASK_START" operation will fail. First, if the account key holder or the optional client key holder has not submitted the information required by the government of a jurisdiction for which a scrub task was submitted, you will receive an error and not be issued a task key. Either the required information may be submitted through an additional registration process, or the jurisdictions that the scrub request is being submitted for may be limited. For more information, see Error Codes 203 & 204 in Section 4.2 of this document. Second, if the SALTA or SALTB values being used by the client application are not current, then you will receive an error. In this case, you will need to submit a "GET_SALTS" operation, receive the new salt values, and rehash the original entries. For more information, see Error Codes 113 & 114 in Section 4.2 of this document. |
This "OP" code is sent to add a file of hashed entries to a particular task. These are the entries to be scrubbed. More than one file may be added to each task. Each file must only contain a single type of entry, however files of different types may be attached to a single task. For example, one task may scrub e-mail addresses, domains, and Yahoo instant messenger IDs, but the task must have a minimum of three files associated with it (one for each type).
Attached files must follow the specifications described in Section 2 of this document. Specifically, they should be byte-wise binary files with the high-nibble first. Each file should be hashed using MD5. The results of the file's MD5 hash should be retained in HEX format as a checksum. This checksum is passed along with the "TASK_ADD" operation.
The file itself should be attached to the post as an HTTPS multipart-form data post (for more information, see RFC 1867). Files must not exceed 40 megabytes. We suggest you limit each file to 2.5 million entries for binary files or 1.25 million entries for HEX files, to remain under this cap. If you need to scrub more than this, you may attach multiple files to a single task. All files attached to a single task will be scrubbed as if they were one request and the results will be returned together.
It is important to store the TASK_ADD_DOMAIN value that is returned as part of the "TASK_START" operation results. All file upload POST operations MUST post to this domain.
Please note that the scrubbing process does not begin until all the files have been uploaded and you submit a "TASK_COMMIT" operation.
Input: | Requires the basic parameters described in Section 3.1 of this document. In addition, you must pass the task key obtained through the "TASK_START" request, the size of the file, and the type of the entries contained in the file. These parameters are explained in more detail below:
|
Output: | In the case of a successful file upload, the resulting XML will be:
<XML>
<REQUEST>
<OP>TASK_ADD</OP>
<TASK_KEY>[32 characters]</TASK_KEY>
</REQUEST>
<RESPONSE>
<TIMESTAMP>[formatted like Sun, 17 Jul 2005 01:51:35 -0400]</TIMESTAMP>
<TRANSACTION>[35 characters]</TRANSACTION>
<RESULT>SUCCESS</RESULT>
<FILE_CHECKSUM>[32 characters]</FILE_CHECKSUM>
<UPLOAD_REQUESTS_LEFT>[remaining number of upload files that can be included with this task]</UPLOAD_REQUESTS_LEFT>
<ISTEST>[If present, you are connected to a test server. This field will contain information about that connection]</ISTEST>
</RESPONSE>
</XML>
The UPLOAD_REQUESTS_LEFT field represents the remaining number of file uploads that can be associated with this task. A maximum of 50 possible files may be associated with a single task. This allows a maximum of approximately 125 million entries checked per task. If you need more than 125 million entries checked per task, please contact the Registry Compliance Customer Support Staff and we will work to accommodate you. There are several probable reasons that a "TASK_ADD" operation will fail. For more information on these possible errors, see Section 5.2 of this document (generally, Error Codes 220-224). |
After all files are uploaded and associated with a particular task, you need to run a "TASK_COMMMIT" operation to begin the scrubbing process. This operation ensures that everything is correctly formatted and computes the estimated cost necessary to complete the scrubbing process for this task.
Input: | Requires the basic parameters described in Section 3.1 of this document. In addition, you must pass the task key obtained through the "TASK_START" request.
|
Output: | In the case of a successful "TASK_COMMIT" operation, the following XML is returned:
<XML>
<REQUEST>
<OP>TASK_COMMIT</OP>
<TASK_KEY>[32 characters]</TASK_KEY>
</REQUEST>
<RESPONSE>
<TIMESTAMP>[formatted like Sun, 17 Jul 2005 01:51:35 -0400]</TIMESTAMP>
<TRANSACTION>[35 characters]</TRANSACTION>
<RESULT>SUCCESS</RESULT>
<TOTAL_FILES>[number of files attached to this particular task]</TOTAL_FILES>
<SUCCESSFULLY_UPLOADED_ENTRIES>[number of entries to be scrubbed]</SUCCESSFULLY_UPLOADED_ENTRIES>
<TYPES_UPLOADED>
<TYPE>
<TYPE_CODE>[type included in the task (e.g., EML, TWT, etc...)]<TYPE_CODE>
<NUM_UPLOADED>[number of entries uploaded for this type]</NUM_UPLOADED>
</TYPE>
[if there is more than one type
associated with a task, the <TYPE> field and contents will be
repeated for each type in a task]
</TYPES_UPLOADED>
<ESTIMATED_FEE>[approximate cost in U.S. dollars to complete task]</ESTIMATED_FEE>
<ESTIMATED_SECONDS>[estimated time to complete in seconds]</ESTIMATED_SECONDS>
<CLIENT_KEY>[optional 64 character key if client is associated]</CLIENT_KEY>
<ISTEST>[If present, you are connected to a test server. This field will contain information about that connection]</ISTEST>
</RESPONSE>
</XML>
Once a "TASK_COMMIT" operation is run for a particular request, that task is closed and you may not add any more files to the task. If you need to cancel a task or check the task's status you may run the "TASK_CANCEL" or "TASK_STATUS" operations, described below. There are several probable reasons that a "TASK_COMMIT" operation will fail. For more information on these possible errors, see Section 5.2 of this document (generally, Error Codes 230-233). |
You may obtain a list of your currently open tasks by using this operation. You may also obtain a list of your tasks, regardless of whether they are open or not, if they have been run in the last seven days. Open tasks are defined as tasks for which a "TASK_START" operation has been submitted, but for which the server has not received a "TASK_COMMIT" request.
Input: | Requires the basic parameters described in Section 3.1 of this document. In addition, you must pass a "status_to_return" parameter to specify whether you wish to limit the request to open tasks or to all tasks started in the last seven days.
|
Output: | In the case of a successful "TASK_LIST" request, the following XML will be returned:
<XML>
<REQUEST>
<OP>LIST_TASKS</OP>
<STATUS_TO_RETURN>["OPEN" or "RECENT"]</STATUS_TO_RETURN>
</REQUEST>
<RESPONSE>
<TIMESTAMP>[formatted like Sun, 17 Jul 2005 01:51:35 -0400]</TIMESTAMP>
<TRANSACTION>[35 characters]</TRANSACTION>
<RESULT>SUCCESS</RESULT>
<RETURNED_TASKS>
<TASK>
<TASK_KEY>[32 characters]</TASK_KEY>
<TASK_STATUS>[current task status (as described below]</TASK_STATUS>
<TASK_PROGRESS_SUMMARY>[description of current status (as described in Section 3.2.7)]</TASK_PROGRESS_SUMMARY>
<TASK_STARTED>[timestamp on which the task was first started]</TASK_STARTED>
</TASK>
[if there is more than one task returned for the query, the <TASK> field and its contents will be repeated]
</RETURNED_TASKS>
<ISTEST>[If present, you are connected to a test server. This field will contain information about that connection]</ISTEST>
</RESPONSE>
</XML>
Possible TASK_STATUS field values include:
Typically, the "TASK_LIST" operation will fail if you are not authorized to use the API, if the client key is malformed, or if other errors occurred during the operation. For more information, see the Error Codes in Section 5.2. |
You may cancel a task at any time. If a task is cancelled it will be closed and no longer open for file uploading or scrub processing. You cannot reactivate a task after it has been cancelled. Uncommitted tasks are automatically cancelled one week after they are first started. Completed tasks that have not been downloaded are automatically cancelled, without the account being billed, 30 days after the task was first started.
Input: | Requires the basic parameters described in Section 3.1 of this document. In addition, you must pass the task key obtained through the "TASK_START" request.
|
Output: | In the case of a successful "TASK_CANCEL" request, the following XML will be returned:
<XML>
<REQUEST>
<OP>TASK_CANCEL</OP>
<TASK_KEY>[32 characters]</TASK_KEY>
</REQUEST>
<RESPONSE>
<TIMESTAMP>[formatted like Sun, 17 Jul 2005 01:51:35 -0400]</TIMESTAMP>
<TRANSACTION>[35 characters]</TRANSACTION>
<RESULT>SUCCESS</RESULT>
<ISTEST>[If present, you are connected to a test server. This field will contain information about that connection]</ISTEST>
</RESPONSE>
</XML>
The "TASK_CANCEL" operation will typically fail if you are not the owner of a particular task, if the task ID is malformed, or if other errors occurred during the operation. For more information, see the Error Codes in Section 5.2. |
To check the status of a task, you may submit a "TASK_CHECK" operation. If the task is complete, the "TASK_CHECK" operation will provide a final cost to complete the scrubbing operation. If the associated account has sufficient funds, the "TASK_CHECK" operation will also provide a code for retrieving the results file. Retrieving the file will charge the given amount from the account associated with the task. If insufficient funds are available, the "TASK_CHECK" operation will provide instructions on how to deposit the requisite funds to complete the task. Section 4 in this document describes in detail how to handle the results file.
The "TASK_CHECK" operation will also return the status of a task if it is not yet complete. Assuming the task has been committed but not completed, the operation will include updated estimates on the time before the task will be complete. If the task is not yet committed, the operation will return its current status as well as additional information about the task.
Input: | Requires the basic parameters described in Section 3.1 of this document. In addition, you must pass the task key obtained through the "TASK_START" request.
|
Output: |
In the case of a successful "TASK_CHECK" operation, the following XML is returned:
<XML>
<REQUEST>
<OP>TASK_CHECK</OP>
<TASK_KEY>[32 characters]</TASK_KEY>
</REQUEST>
<RESPONSE>
<TIMESTAMP>[formatted like Sun, 17 Jul 2005 01:51:35 -0400]</TIMESTAMP>
<TRANSACTION>[35 characters]</TRANSACTION>
<RESULT>SUCCESS</RESULT>
<TIMES_DOWNLOADED>[the number of times the task results have been downloaded]</TIMES_DOWNLOADED>
<TASK_STATUS>[current task status]</TASK_STATUS>
<TASK_PROGRESS_SUMMARY>[description of current status]</TASK_PROGRESS_SUMMARY>
<TOTAL_FILES>[number of files attached to this particular task]</TOTAL_FILES>
<SUCCESSFULLY_UPLOADED_ENTRIES>[number of entries to be scrubbed, regardless of type]</SUCCESSFULLY_UPLOADED_ENTRIES>
<TYPES_UPLOADED>
<TYPE>
<TYPE_CODE>[type included in the task (e.g., EML, TWT, etc...)]<TYPE_CODE>
<NUM_UPLOADED>[number of entries uploaded for this type]</NUM_UPLOADED>
</TYPE>
[if there is more than one type
associated with a task, the <TYPE> field and contents will be
repeated for each type in a task]
</TYPES_UPLOADED>
<TASK_ADD_DOMAIN>[domain to which all subsequent TASK_ADD operations should be posted]</TASK_ADD_DOMAIN>
<UPLOAD_REQUESTS_LEFT>[number of file upload requests left for this task]</UPLOAD_REQUESTS_LEFT>
<ESTIMATED_FEE>[approximate cost in U.S. dollars to complete task]</ESTIMATED_FEE>
<ESTIMATED_SECONDS>[estimated time to completion, in seconds]</ESTIMATED_SECONDS>
<SCRUB_FEE>[cost in U.S. dollars to complete/download task]</SCRUB_FEE>
<SUFFICIENT_FUNDS>[whether or not there are sufficient funds in the account to complete/download the task; valid values are: YES or NO]</SUFFICIENT_FUNDS>
<BILL_ESP>[whether or not the funds should be retrieved from the ESP account instead of the client's; valid values are: YES or NO]</BILL_ESP>
<CLIENT_KEY>[optional 64 character key if a client is associated with the task]</CLIENT_KEY>
<RESULT_KEY>[32 character key necessary to retrieve the final results; only available when the task status is FINISHED: CLOSED.]</RESULT_KEY>
<ISTEST>[If present, you are connected to a test server. This field will contain information about that connection]</ISTEST>
</RESPONSE>
</XML>
Some of the XML fields above will not appear depending on the status of the task. For example, if the task has not been committed then there will not yet be an estimated time before it is completed. Once a task has finished scrubbing, the RESULT_KEY field will include a code for downloading the task's results through a "TASK_RESULTS" operation (see below). Possible TASK_STATUS field values include:
These status fields are further explained by the TASK_PROGRESS_SUMMARY field. Possible values for this field include:
Typically, the "TASK_STATUS" operation will fail if you are not the owner of a particular task, if the task ID is malformed, or if other errors occurred during the operation. For more information, see the Error Codes in Section 5.2. |
After a task is complete, you will be issued a result key through the "TASK_CHECK" function. You can use this result key to retrieve the match results.
The task's user account will be charged the total amount for the scrubbing task the first time you access the task results with the result key. You may subsequently access the results after initial download without being charged.
Input: | Requires the basic parameters described in Section 3.1 of this document. In addition, you must pass the task key obtained through the "TASK_START" request and the result key optained through the final "TASK_CHECK" request.
|
Output: | The output of the "TASK_RESULTS" operation contains three key XML parts: SCRUB_REPORT, SCRUB_RESULTS, and POSSIBLE_SCRUB_EXCEPTIONS. These sections are described in detail in Section 4 of this document. The "TASK_RESULTS" operation may be longer than other XML documents and may take longer to download. |
After a task is complete, you will be provided with a result key for your match file through the "TASK_CHECK" operation. To retrieve the results of your scrub process, issue the "TASK_RESULTS" operation described in Section 3.2.8 of this document. The results of your scrub process for all jurisdictions and all types are returned in a single XML document. The XML document also includes report information for the particular task. The XML document will remain online and retrievable via the "TASK_RESULTS" process for at least 30 days after you initially began the task that created it.
Accessing of the results document via "TASK_RESULTS" is limited to the approved IP addresses of the API access key associated with the task. The task's account will be deducted the cost of the scrub when you access the results via the "TASK_RESULTS" operation. This cost was reported exactly via the "TASK_CHECK" operation through which you received the URL to download the file. If insufficient funds are in the account at the time of an attempted file download, you will receive an error.
In addition to the REQUEST section, there are three major sections of the XML results document: SCRUB_REPORT, SCRUB_RESULTS, POSSIBLE_SCRUB_EXCEPTIONS.
Each results document will include a transaction report section. This file will include XML with details on the transaction you submitted. The XML transaction details may include the fields in the example below:
Particular license restrictions on how the results data may be used may be included as part of this document. Specifically, these licenses limit your storing, transferring, reverse engineering, or using these files for any purpose other than removing the entries they represent from a particular client's list. Links to these license terms are included in the document, and you must have agreed to these terms prior to accessing the scrubbing mechanism.
To ensure compliance with each jurisdiction's law, the next scrub of the entries that remain on the list after the current scrub must take place before the GOOD_UNTIL date listed for each jurisdiction.
The Match Results Section of the document describes the "matched" entries that must be removed from a sender's list in order to be in compliance with the law. The matches are divided by jurisdiction. Each jurisdiction is then divided by type. If multiple types were scrubbed at one time, then they will be listed one after another. You are required, under the law, to remove any results listed in this section unless there is a domain exception, as described in Section 4.3 below.
Two MD5 hashes in HEX format are returned for each match. The SALTA match is the same as the hashed entry that you submitted to be scrubbed, however it is in HEX, not binary, format. It is created using an MD5 function of the following format:
A second MD5 HEX hash code, SALTB, is provided as a double-check. This eliminates the already-unlikely chance that a "collision" could result in an entry being removed that is not in fact a match.
To use the double-check mechanism, you should scan your list for any entries that match any of the SALTA_MATCH fields. Once found, you should use an MD5 HEX hash function to rehash the plain-text entry using the SALTB value. This function should look like:
See the following example:
EXAMPLE 4 | |
Type: | E-Mail Address |
Entry: | john.doe@example.com |
SALTB: | LSKJDF093@#SWDKJ^WERLKJ34@#234WEFLKJAER-234LKEJAWIJ34=AW34JLJER-32=+aosiKASLDJ93qiuajsdalj3ijadalksjd983lkjnaskdnlkqwelijasd-324$!23!23 |
MD5 HEX Input for SALTB Double-Check: | EMLjohn.doe@example.comLSKJDF093@#SWDKJ^WERLKJ34@#234WEFLKJAER-234LKEJAWIJ34=AW34JLJER-32=+aosiKASLDJ93qiuajsdalj3ijadalksjd983lkjnaskdnlkqwelijasd-324$!23!23 |
Under the laws, you must remove any entries from your originally uploaded list that match both the SALTA_MATCH and the SALTB_MATCH, unless a domain exception is listed later in "Match Possible Exceptions" section of the document, as explained in Section 4.3 of this document. Continuing to send messages to any matched and verified entry (verified using the SALTB hash) is a violation of the law. You may not store or share the results you receive through the scrubbing process with any party other than the client for which the scrubbing was done. Storing the results or sharing them beyond the client may constitute a felony, punishable by up to 3 years in jail depending on the jurisdiction. Please ensure when constructing your application that you do not violate these storage or sharing provisions of the law. For more information on these requirements, see:
If there were no matches for a particular jurisdiction and type then the NUM_MATCHES_FOR_THIS_TYPE field will contain a "0" (zero) and will not be followed by a <RETURNED_MATCHES> section. If you receive a completely empty file, or if you receive no results file, then an error has occurred.
For domain entry types (DMN), you must remove all e-mail addresses that are created with a matched domain. This requires you to not only map the SALTA_MATCH and SALTB_MATCH back to a particular domain, but to also associate that plain-text domain with any e-mail addresses constructed with it that appear on your list. The legal effect of a domain appearing as a match is the same as if all the e-mail addresses on an uploaded list constructed with that domain were returned as matches.
In certain cases, there will be exceptions from the matched entries that appeared above. At this time, the only case in which this may occur is if a domain was matched but a particular e-mail address under that domain was exempted from coverage. Although exceptions are likely to be rare, you should still construct your systems to accommodate them.
Any exception listed in this section of the document trumps any listing that appears in the SCRUB_RESULTS section. In other words, exceptions may remain on an uploaded list even if there was an indication of a match in the Scrub Results section.
Exceptions will be presented in the following XML format:
Just like matches, exceptions are returned as MD5 HEX hashes, formed with both SALTA and SALTB. Only if an entry on a list hashes to both the SALTA_EXCEPTION and the SALTB_EXCEPTION values may you continue to include it on your list.
Please be aware that just because exceptions appear in this section does not mean they will appear on your list. For example, domain owners may list exceptions for any registered domain. If a registered domain matches an entry on your list, then you will be receive the complete list of exceptions for that domain, even if none ever appeared on your list. This means there does not need to be any correlation between the listed exceptions and the entries on your scrubbed list. Typically, even if exceptions are listed in the POSSIBLE_SCRUB_EXCEPTIONS section, they will not appear anywhere on your scrubbed lists.
As with any other data returned from the Registry Compliance system, you may not share, distribute, store, or attempt to decompile any of the returned exceptions except as necessary to comply with the law. Doing so may constitute a felony under the law. For the specific law requirements, please visit
You may encounter error codes when running a transaction. These codes are all formatted as a 3-digit number. The first digit determines the general class of the code:
In section 5.2 of this document, the specific error codes are outlined. Accompanying each error code is a description of the error. Applications designed to interface with the Registry Compliance API should be aware of these errors and tested to ensure they can respond appropriately.