Home |
Accounts |
Credentials |
Peers |
Projects |
Upload |
De-duplicate |
Cluster |
Tag Clouds |
View |
Browse |
Search |
Buckets |
Datasets |
Assign |
Notifications |
Toolbox |
Code |
Bookmarks |
Validate |
Report |
FAQ |
Service Levels |
Ideas for PCAT Improvements |
PCAT Wiki ToDo List |
ContactPCAT project managers can review and validate the work of coders¶
During training sessions, you may want to review and validate the work of coders to make sure they understand the requirements of the classification task. The original CAT Toolkit was built primarily for this purpose. Adjudication by consensus or expert validation is a key feature of this platform.
To begin validation, the dataset must be locked. To lock a dataset, select View Datasets from the Datasets drop-down menu at the top of the screen. Click on the name of the dataset, then select Lock Dataset at the bottom of the right sidebar. Once the dataset is locked, select “Validate Dataset” from the “Validations” drop-down menu. Select a dataset from the drop-down list to begin. At this time you can also select which specific codes you wish to validate if you do not wish to validate all codes.
You will be shown a document as well as the code(s) selected for it. To validate, either click the Valid button on the bottom of the screen or type 1 or Y. If the coding is invalid, click Invalid or type 3 or N. Using keystrokes is efficient when validating large datasets, but you can use whichever method you prefer without changing the result. After validating the document, the next document will load automatically.
Editing Validations
If you make a mistake when validating, you can still fix it. Select View/Edit Validations from the Validations dropdown menu at the top of the screen. Click on the name of the dataset you want to edit, then you can select which coders or codes you want to review. Click on the
icon to change your validation.
Coder/Dataset Statistics
Coder Statistics - This tool shows the coder's individual statistics. By choosing a coder, PCAT will provide information about them in the following order:
- # Datasets - Total number of datasets the coder has worked on
- # Annotations - Total number of codes applied by the coder
- # Validations - Total number of validations made by the coder
- % Correctness - The total percentage of occurrences where the coder's annotations was a valid answer
Dataset Statistics - This tool shows statistics for a specific dataset. By choosing a dataset, PCAT will provide information in the following order:
- Total Valid Answer - The total number of valid answers made by all coders
- Validations by coder - Total number of valid answers made by each coder
- Validation by code - Total number of valid answers for each code
Most Frequently Asked Questions
Why would I use this system? |
Where do I get FDMS bulk downloads? |
Does PCAT identify duplicates? |
What is QDAP?
© 2009 - 2010
Qualitative Data Analysis Program (QDAP), in the
University Center for Social and Urban Research, at the
University of Pittsburgh, and
QDAP-UMass, in the
College of Social and Behavioral Sciences, at the
University of Massachusetts Amherst. As of 2010, PCAT and this PCAT Help Wiki are maintained and improved by personnel from
Texifter, LLC, which is a software start-up located in North Amherst & Springfield, MA and online at
http://texifter.com/.
Content on this website was made possible with the following grants from the National Science Foundation:
III-0705566 "Collaborative Research III-COR: From a Pile of Documents to a Collection of Information: A Framework for Multi-Dimensional Text Analysis" and
IIS-0429293 "Collaborative Research: Language Processing Technology for Electronic Rulemaking." We are also grateful for financial support from the U.S. Environmental Protection Agency and the U.S. Fish & Wildlife Service. **Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the National Science Foundation.**
==
==
Home |
Accounts |
Credentials |
Peers |
Projects |
Upload |
De-duplicate |
Cluster |
Tag Clouds |
View |
Browse |
Search |
Buckets |
Datasets |
Assign |
Notifications |
Toolbox |
Code |
Bookmarks |
Validate |
Report |
FAQ |
Service Levels |
Ideas for PCAT Improvements |
PCAT Wiki ToDo List |
Contact