WISEdata Portal Home Screen: Data Pipeline Status Overview
This mini-tutorial will provide a ‘top-to-bottom’ tutorial for the WISEdata Portal, Agency Tab, Home screen.
It will cover all of the features presented on the Home screen, and provide insights for when, and how customers can get the most impact from these features.
The Home screen displays an overview of your data flow status, as your data flows through the "data pipeline" from your LEA’s student information system (SIS) into WISEdata Portal. You will see statistics about whether or not data made it out of your SIS to the Ed-Fi API and from there into the WISEdata Portal.
Click the links below to navigate to different sections of this page:
Alerts
At the very top you may see an orange banner, alerting users of:
- System alerts
- Unacknowledged data alerts
- Other alerts
Clicking the blue “Learn More” button will bring you to the Alerts screen. Read the WISEdata Portal Alerts Mini Tutorial for more details.
Validation Message Summary
Just below the Alerts (if any are displayed) is the Validation Message Summary section of the Home screen. This section gives a quick visual of the health of your submitted data. If you have errors, you will want to navigate to the Validation Messages screen to begin reviewing and resolving issues.
These are ‘L2’ errors and warnings, which are different from the L1 errors displayed in the Data Pipeline Stats section of the Home screen, and the WISEdata Step 1 (API) and Step 2 (WISEdata Portal) sections of the Home screen.
L2 validations indicate that your data has passed into WISEdata Portal (through the Ed-Fi API), but that the data isn’t what is expected, is invalid or inaccurate in some way. All L2 validations are coded with a four-digit code, and each one is linked to a Knowledge-based Article (KBA). The KBA describes the nature of the error or warning, the logic guiding the validation, steps on how to correct your data and an example use case/scenario.
Three long and narrow horizontal bar graphs display the “Distribution of Messages” for your LEA’s errors (displaying in red), unacknowledged warning (displaying in orange) and acknowledged warnings displaying in green).
Multiple school years (current school year on top, and two prior years below) will display by default.
Hovering your mouse over the differently colored sections on these bar graphs will display a same-colored ‘bubble’ that gives a numerical value for that type of validation.
Clicking anywhere on the bar graph, will change your screen to the Validation Messages Screen.
Read the WISEdata Validation Messages Mini Tutorial and the Acknowledging Warnings in WISEdata Portal Mini Tutorial for more details.
Check out your validation totals for errors, acknowledged warnings, and unacknowledged warnings at the bottom of the home page.
Data Pipeline Status - Vendor(s) Section: Gauging Successful Data Transmission
When flowing correctly, the data you enter into your SIS is submitted to WISEdata for state reporting to DPI.
You will want to ensure it is transmitted successfully. Best practice is to build a habit of regularly checking the Data Pipeline Status first before navigating to the other tools in WISEdata Portal. This makes you aware of any data flow issues.
If the data in your SIS is not flowing to WISEdata Portal, the work you do to push over accurate data from your SIS will not be successful if data is not properly flowing.
There are graphics on the Home screen in WDP to help you know if this is happening: data Data Pipeline Status is one of them.
Multiple school years (current school year on top, and two prior years below) will display by default. You can also mark or unmark a checkbox to view a particular school year if you wish to narrow down your data review.
Clicking on the ‘information’ icon in the headers, Data Pipeline Status and Vendor(s), will make the “Vendor/WISEdata Communication Status” information box either appear or disappear:
Include API Read (GET) Transactions
You will notice a checkbox in the Vendor(s) section, Include API Read (GET) transactions, located below the information box/at the top of your listed vendor tool tiles. If you need a refresher on what Ed-Fi is, please check out the Ed-Fi Credential Application Mini-Tutorial.
SIS Vendor Tool Tile Displays
When you check this box (checkbox turns blue), instead of seeing “No data received” within the tile under the vendor tool name, you will see a date/time stamp. This date/time stamp tells you the last time a transaction occurred between the vendor tool and the Ed-Fi API Credential application. The screenshot below displays:
- A grid of gray tiles.
- This example has three rows, each with three tiles: school year, SIS vendor tool 1, SIS vendor tool 2.
- Each tile has a darker gray band across the top naming either:
- The school year, or
- The name of the SIS vendor tool.
- Each tile has a lighter gray bottom section
- The lighter gray bottom section shows a date/time stamp for the “Last request to WISEdata API.”
- The leftmost column of tiles will always display the school year
- Current school year on top, and the
- Last two prior years listed below, chronologically.
- The tiles will play horizontally to the right, across each school year, list one tile for each SIS vendor tool for which an LEA has a subscription.
- From the Vendor(s) section you can check to see if some or all of the data that should've pushed from your SIS in the past 24 hours was submitted or if a Level 1 (L1) error rejected your data outright.
-
Each SIS vendor tool tiles states the name of the vendor tool, displays an information icon, a date/time stamp of the last transaction, shows the status of your data flow over the past 24 hours by using a color-coded indicator:
-
A green checkmark, meaning that your data is flowing successfully from your SIS vendor tool, through the Ed-Fi API, to DPI’s WISEdata Portal
(Example: Vendor Tool 1)
-
A red “X”, meaning Level1 errors are interrupting data flow
(Example: Vendor Tool 2)
-
-
These color-coded thresholds are set by DPI to alert you to potential issues with your SIS pushing data to us.
- What Does a Green Checkmark mean? (Example: Vendor Tool 1)
-
If you see green for your SIS, your data was submitted successfully.
Hovering over the checkmark will display a confirmation message:
“None of the transactions in the last 24 hours submitted by ‘Vendor Name’ for the ‘current school year’ are in error.”
- What Does a Red “X” Mean? (Example - Vendor Tool 2):
-
A red “X” indicator means that there are errors preventing some/all of the data from flowing as expected through the Ed-Fi API. This lack of data flow between your SIS vendor tool and the Ed-Fi API is an “L1” error.
However, unless you are actively troubleshooting a data flow issue it is not necessary to prioritize L1 errors. If no data flow issues exist, it is a better use of time to concentrate on L2 errors.
If you are actively troubleshooting a data flow issue, then L1 errors can be extremely helpful, and you should provide this information to your SIS vendor support representatives.
There are L1 errors issued from the Ed-Fi API that may prevent your data from transmitting. Some examples of this API data prevention could be:
- you're no longer the primary enrollment-holding school or
- a crucial program record is missing when trying to send a crucial data record (e.g., overlapping enrollments, program data, etc.).
It's possible there are also other issues with your SIS preventing proper submission, so it's important to keep an eye on this step. This connection must be fixed by communicating with your SIS vendor tool support staff. DPI cannot provide support for SIS vendor tools.
Hovering over the checkmark will display an information message: “One or more transactions in the last 24 hours submitted by ‘Vendor Name’ for the ‘current school year’ are in error.”
To troubleshoot which L1 errors occurred, click the red “x” on the tile you want to drill into. This will open up a new browser tab for the WISEdata Portal Home screen.
This new screen will display the name of the vendor tool you selected for the year you selected. A table will display a list of up to 100 L1 errors.
Transactions will display sorted by Aggregate Name, such as Student Program Association.
You will also see the HTTP Method, such as POST (add data) or PUT (update data), which explains how your SIS was interacting with the API. In addition you will see the error and success counts with a timestamp for when the last transaction was processed.
The table also reveals error counts and success counts.
-
Error Count: If you see 0 in some areas in this column, this is good, as it indicates zero errors. However, a word of caution that a lack of errors does not prove connectivity between your vended application to the Ed-Fi API. Successful connectivity can only be affirmed by one’s Success Counts. The topics with errors will provide insights for why you are not seeing data flow in unique situations.
-
When the Error Count is not zero, and you need to troubleshoot a known issue, please do this by contacting your SIS vendor tool support persons. The DPI Customer Services team cannot troubleshoot or problem solve SIS vendor data flow issues.
-
With the support of your SIS vendor tool support representative, you can find additional details by clicking the expander tool from the far left column (blue, rotating triangle) for a detailed view. You can even view the nitty-gritty technical information by clicking the ‘View JSON’ button. It may be very helpful to copy the JSON text and hand it off to your IT person or SIS vendor support when technical assistance is needed.
-
-
-
Success Count: Seeing 1 ‘Success Count’ is an indicator that the connection between your SIS vendor tool and the Ed-Fi API exists. If you see all zeros in all areas under this column, this indicates ‘Zero success' which is a problem. Zero success count means there is an issue with data flow, specifically your data is not flowing through the Ed-Fi API. Since data flows from your vendor application, to resolve the issue start by reaching out to your vendor.
There are L1 errors issued from the Ed-Fi API that may prevent your data from transmitting. Some examples of this API data prevention could be:
- you're no longer the primary enrollment-holding school or
- a crucial program record is missing when trying to send a crucial data record (e.g., overlapping enrollments, program data, etc.).
It's possible there are also other issues with your SIS preventing proper submission, so it's important to keep an eye on this step. This connection must be fixed by communicating with your SIS vendor tool support staff. DPI cannot provide support for SIS vendor tools.
WISEdata Section
Step 1: WISEdata API: Did Your Data Make It into WISEdata?
In the WISEdata section of the portal's home page, you can see the last time certain types of records were accepted, such as the last time a student enrollment record was sent or the last time a special education record was sent. The various time stamps and color coding will help you see how fresh your data is.
Clicking on the ‘information’ icon in the headers, Step 1: WISEdata API, will make the “WISEdata API Database - Data Last Received” information box either appear or disappear:
From the API your data undergoes validation and then pushes into the WISEdata database. Sometimes, though, data will stall in the API for various reasons instead of pushing into WISEdata for validation.Ed-Fi API acts like the 'middle man' that transfers data from your SIS into data DPI's WISEdata Portal can understand. It allows two different languages to speak to each other. Again the red “X” indicator means that there are critical errors preventing some/all of the data from flowing as expected through the Ed-Fi API. This lack of data flow between your SIS vendor tool and the Ed-Fi API is an “L1” error.
Thresholds for data quality (defined by ‘logic’) are set by DPI. Each data category (enrollment, program, discipline, etc.) will display with a red, yellow, or green icon and a date/time stamp similar to the Vendor(s) section of the data pipeline. The data and time show when that type of data record was last received.
-
A green icon (example: 2022-23 Roster tile) means your data is fresh, received within the past 10 days for the current school year.
-
Thresholds are larger for previous school years, e.g., 20 days for last school year.
-
-
A yellow icon (example: 2021-22 Program tile) suggests your data is fairly fresh but has not come in as recently as the green timeframe.
-
A red icon (example: 2023-24 Parent tile) alerts you that no data has been received in the past 10 days and may be stuck in your SIS or the API. If you're seeing red icons, check with your SIS vendor to see if certain data endpoints were turned off or need a manual sync when switching between school years.
Even if data was received, if the transmission time listed seems longer than you'd expect, drill into the particular data set(s) with the issue by clicking on the color-coded indicator.
This opens up a new tab in your browser, displaying the data statistics for when each type of record was last sent from the API. This is useful information for troubleshooting and letting you know if validation is still needed
Step 2: WISEdata Portal: Has Your Data Been Imported from the API and Validated?
The second step of the WISEdata section shows whether or not the data in the API and in the WISEdata Portal are in sync. If you see green icons, your data is in sync.
Clicking on the ‘information’ icon in the headers, Step 2: WISEdata Portal, will make the “WISEdata Portal Database - Data Last Received” information box either appear or disappear:
Queue Import and Validation
An import and validation cycle runs each night for all the data. However, you can trigger validation manually if:
- your data has stalled in the API, or
- substantial changes were made to your data and you want to validate right away instead of waiting for the nightly validation cycle.
Simply push Queue Import & Validation to place your data in line for validation.
- The length of time for validation will vary based on how many districts are in line in front of you.
- If you try to queue validation when it's already running, you will see a pop up box to alert you that the import is in progress.
Request Notification When Queuing Validation
If you wish to receive an email notification when your validation cycle completes, mark the checkbox below the validation button and verify or update the email address that pre-fills in the box.
The email will only send for this validation, not all validations and not for system-wide validation jobs that run each night. You must mark the checkbox prior to pushing Queue Import & Validation.
As seen in all other sections of the Home screen, the color-coded icons should alert you if your data is or isn't in sync. If you are seeing yellow or red icons, your data needs importing and validation. You can drill into each data category by clicking the color-coded icon in each tile to see what values look like in the API.
NOTE: Record counts are listed for three data areas, but record counts do not necessarily align to the number of students. It is possible for one student to have multiple records in a data category. As such, the record counts should not be used as student counts.