If you have a question that is not answered here, please email Todd.OBrien@noaa.gov for help.
Todd loves to hear from you, and he wants to improve this FAQ or the Toolkit's ability to auto-detect errors by itself.
Using COPEPOD's "Failed Data" Report:
If your uploaded data fails during the processing step, you should see this screen:
The "Click here to view the data format report Link" will take you to a report that reports any problems related to column headings, mixed up date formats, bad values, and more.
The most common date errors are from a bad date column heading (e.g, "DATE-MDY") not matching the date values (e.g., they may be "DATE-DMY").
This section will give some basic statistics on value ranges that can be used to detect errors (e.g., your data from 1975 to 2015 may have a minimum year of "1957" (or a bad month value of "21" because of a mistyped value in the date column).
The "Show me the CSV translated table format Link" will show you how the tables and columns of the data file will be seen by COPEPODITE.
This can be useful for detecting problems in the CSV format due to commas in the label, foreign characters, or when Excel is actually creating a semicolon-separated-file (;;;;;).
If the table layout found in this screen is goofy or messed-up, COPEPODITE won't be able to process the file.
The "Show me the uploaded CSV file (raw view) Link" will show you a raw text version of what was actually uploaded.
This can be useful for detecting if your version of Excel is creating a semicolon-separated-file (;;;;;) or if non-standard ASCII characters are causing the file to cut-off during the upload processing.
|
|
"Unreadable Data File" Errors:
"I received an email saying COPEPODITE failed to process because of an unreadable data file."
COPEPODITE can only read plain text (".txt") or comma-separated-values (".csv") files.
COPEPODITE can not read Excel binary files.
If you try to upload an Excel ".xls" or ".xlsx" file, the program will not be able to read it and will return this error.
COPEPODITE also expects the very first row of the ".csv" or ".txt" data file to contain COPEPODITE column labels, and the very first column of this file must be a "DATE" column. (The first five letters of your text/csv data file will therefore be "DATE-", as is "DATE-MDY" or "DATE-YMD" or "DATE-DMY".) If anything other than a DATE column is found here, the program will also give this error.
|
Some or All of my Data Values are missing:
"In my result plots, I noticed that some (or all) of my data values were (randomly) missing."
If data are missing in your results, the first place to look is in the "Data Format Report" link that is provided in the "Data Processing" module of your COPEPODITE results.
COPEPODITE has problems reading numbers with non-numeric pieces. For example, the number "1.77e10" may be flagged and skipped by COPEPODITE because of the "e". COPEPODITE may or may not read this correctly depending on how your software formats the scientific notation values. Your safest best is to not use scientific notation at all. (Note that COPEPODITE also currently has a 9-digit number limit. If you are working with microbial plankton counts, select a base unit ("number per ml" versus "number per cubic-meter") that keeps the count values less than 10,000,000 (1E10).)
COPEPODITE does NOT like commas in its numbers. The "thousands place" comma ("1,000" vs "1000") will cause the number to be flagged as "non-numeric" and skipped by the software. In a case like this, any value greater than 999.999 would be missing in your plots. It may also confuse the CSV file format and cause data to shift into other columns.
COPEPODITE ignores any text provided in the number fields. This allows you to make comments ("bad sample") in a data cell. It also means that trace value indicators ("<1", "trace") or any sort of flagging ("1.25*") will also be ignored. If you want to represent trace values, use a numeric representation ("0.001").
|
European vs. United States "Excel" Date Formating:
"I am having problems working with data from another country. The dates keep messing up and changing."
|
European and (some?) Canadian versions of Excel appear to default dates to "day-month-year" format.
United States versions of Excel appear to default dates to "month-day-year" format.
If you open up a CSV file (or copy and paste from an Excel file) containing a European date format into a United States version of Excel, the "day-month-year" dates will be partially and incorrectly converted as if they were "month-day-year". What this means is that any day equal of less than "12" will be read as a month (i.e., 03/12/2005 is read as March-12-2005, even though the intended data was December-03-2005), and the remaining dates will be left as "junk text". The end result is that the date column will be completely messed up.
Solution One: If you have access to the binary (".xls") Excel file, select the (foreign) date column(s) and set the formats of those columns to "date" format (versus "general" format). (The point of doing this is to make sure Excel knows these are "date values" rather than general content values.) Now you should be able to safely copy and paste the dates into the other country's Excel version.
Solution Two: If you only have a CSV file, open that CSV file with a TEXT editor (i.e. Windows "notepad" or "wordpad") and replace all of the "/" or "-" date separator characters to "_" (e.g., "03/12/2005" or "03-12-2005" become "03_12_2005"). This will make Excel handle this column as text and it will not try to convert the dates. COPEPODITE will still be able to correctly read in the date information.
|
|
