This article describes how you can configure SimplyDo to handle idea assessments. It includes:
How to set up assessments, assign maximum scores, weightings, and assessor instructions
How to invite assessors to assess ideas
How assessors can make their assessments
How to view the assessment results
Please remember that your SimplyDo account manager is always on hand to provide assistance and demonstrations to help you learn more about the platform and its features.
Idea assessments are a great way to crowdsource insights from one or more assessors on ideas in your challenge. Scorings are made by individuals on a per-assessor and per-section basis.
Typical use cases for assessments include:
Judging for ideas competitions;
Assessment as part of a procurement process;
Due-diligence processes, with assessors from across various departments;
Evaluation of potential suppliers;
The general flow for handling assessments in SimplyDo proceeds as follows:
A challenge manager configures the scoring and assessment scheme, including weightings and instructions, in the idea format for their challenge.
The challenge manager invites assessors to assess ideas.
Assessors complete their assessments. Single assessors can assess multiple ideas.
The challenge manager can view the individual and aggregated scores from all assessors, and action accordingly.
2. Useful notes
Please note the following:
Assessments are set-up by an administrator or challenge manager, and exist on a per-challenge basis (tied to the idea format);
Assessors can be existing SimplyDo users, or invited especially for the assessment process;
Assessors can not view the scores or notes made by other assessors (unless they themselves are an administrator);
Assessors must submit their assessments before the assessment deadline, which is set by the challenge manager.
Assessors will be able to view ideas even if the challenge’s settings are restricted. As such, make sure that your assessors are qualified to view the ideas in-line with your challenge strategy.
3. Setting up an assessment scheme
The first stage is for the challenge manager to configure how the assessments will run. This includes:
Choosing a mark or scoring scheme for each section of the idea format;
Writing instructions to assessors;
Choosing a deadline for assessments to be made by.
In this section we’ll assume that you already have a challenge setup, and an idea format has been selected. In the examples below, we’ll use the default SimplyDo 3 Is format, though the concepts are the same no matter which format you use.
3.1. Enable assessments for your challenge
Navigate to your challenge in the SimplyDo platform. Once on your challenge, click the “Edit format” button to view the format editor for your challenge.
Enable assessments for the challenge by checking the Enable idea assessment checkbox.
A number of additional fields will now be available in your format.
3.2. Set an assessment deadline
After checking the Enable idea assessment checkbox (as described in section 3.1), you’ll notice an option to enter a deadline for the assessment.
An assessment deadline is a date after which assessments cannot be modified. Any assessments made until the deadline date will be the ones taken forward for consideration by the challenge manager.
To set an assessment deadline, choose a date in the Deadline field. The deadline will be saved for you automatically.
Please note: the assessment period can also be closed at any time using the Close assessment button.
After the assessment period is closed (either by closing manually or after the deadline has passed), new assessments cannot be made or edited.
3.3. Configure an assessment scheme
An assessment scheme consists of a per-section maximum score, a weighting for each section, and notes that provide instructions to the assessors.
At the top of each section you will notice a Section assessment block (assuming you have assessments enabled as described in section 3.1).
Complete this block as follows:
The total score indicates the total available marks for this section. For example, if 10 is selected, then assessors can provide a score out of 10 for this section.
The score weighting is a percentage of the entire format that indicates what the marks in this section should comprise. For example, if a weighting of 50 is provided, then the section will form half of the score for the whole idea. The weightings across all of the sections must, when summed, equal 100.
The notes field allows you to input assessment notes or instructions to provide context or guidance for your assessors.
Repeat the process for the other sections in your format. Note: in the image above, there is only one section for the format and so this section must have a 100% weighting.
Important: Once done, remember to save your format using the Save format button.
Your assessment scheme is now complete and you are ready to invite and assign assessors.
4. Assigning assessors
Once your assessment scheme has been setup (as described in section 3 above), you are now ready to assign assessors.
SimplyDo is flexible and allows you to assign different assessors to different ideas, and each idea can have multiple assessors. This means that should you have lots of ideas, you can spread out the workload whilst keeping things fair with multiple assessor points of view.
4.1. Find the “Assign assessors” tool
Assessors can be added using SimplyDo’s “Assign assessors” tool. To find this, navigate to the Ideas tab of your challenge.
Assuming you have ideas already in your challenge, you can open the tool by clicking the settings icon next to the idea you want to assign, and then select Assign assessors.
Alternatively, you can assign assessors to several ideas in one go by selecting them in the list, and then using the dropdown selector.
Whichever method you choose, you should now see the following screen.
4.2. Add a user already registered on SimplyDo
If the assessors you want to assign already have an account on SimplyDo (i.e. they are a member of staff in your organisation), you can simply search for them by name or email address on the Search tab.
4.3. Add a user not yet registered on SimplyDo
If the assessors you want to assign does not already have an account on SimplyDo (i.e. they may be external to your organisation), you can enter their email address on the Invite tab.
4.4. Complete the assignment
As you continue to add assessors (using either the search or invite method, described in sections 4.2 and 4.3 above), they will be added to the chosen users section.
When you are ready, click the Assign these users button to complete the assignment.
The selected assessors will be sent an email inviting them to access the platform and complete their assessments.
Back on the ideas tab of your challenge, you can view the status of the assessors for each idea by selecting the assessment overview information shown on each idea (as shown in the image below).
5. Removing assessors
At any time you can remove assessors from an idea. Once completed, the assessor will no longer be able to create assessments against the idea, and any scores added so far will be excluded from calculations.
To remove an assessor, first navigate to the ideas tab of your challenge. On the assessment overview information (as shown in section 4.4 above) expand the block for your chosen assessor, and then click the remove as assessor button.
6. Completing an assessment
Now that we’ve seen how to set-up assessments and assign assessors, the assessors can now go and complete their assessments.
After receiving their invitation emails and registering or logging-in to the SimplyDo platform, assessors can find their “to-do” assessments on the top-right of their homepage.
Alternatively they can find the challenge, if they are in the audience, via the challenges page.
The assessor will then be taken to the assessment dashboard for the challenge. On this screen, the ideas that the assessor has been assigned to in this challenge will be displayed. The deadline for assessment submission is also shown.
To begin an assessment, the assessor can select the view idea button next to the idea they want to assess. This will bring out the idea for them to view.
The assessor can then read through the idea.
Below each section an assessment block is displayed. Here, the assessor can enter a score and notes (e.g. for justification). Also displayed are the assessor instructions we wrote earlier when setting-up the scoring scheme.
Once the assessor is happy with their assessment, they can return to the list by clicking “off” the idea pane or by using the close preview button shown at the top of the idea
Once back on the assessments dashboard, the assessor can view the scores and status for each of the ideas they have been assigned to.
When the assessor is happy with all of their assessments, they can submit their assessments. We recommend that assessors initially draft all of their scores and notes for each idea, and then return to them again to ensure they believe their assessments are fair.
Assessments can be submitted one at a time, or all in one go by using either the submit assessment button shown alongside each idea, or the submit all assessments button shown at the bottom.
Once submitted, the status label for each idea will update accordingly.
Assessors can also export their assessments as an Excel spreadsheet by clicking the download summary report button.
Please note that assessors will not be able to alter or submit their assessments after the assessment deadline has passed.
7. Viewing assessments
The final stage in the process is to view the assessments made by your assessors.
7.1. View assessments on the ideas tab
Assessments can be viewed by navigating to the ideas tab of your challenge. On this page, an assessment summary is displayed for each idea. For example, in the image below we can see that the current submitted score is 75% and that only one of the two assigned assessors has submitted their assessment for this idea.
Clicking the assessment summary will bring up the assessment report for the idea. On the report, you can expand each assessor separately to view a per-section score breakdown, as well as any notes left by the assessor.
From this view, you can also unsubmit an assessment by clicking the unsubmit button. This will allow the assessor to update their assessments and re-submit.
7.2. View assessments on the project board
If you use project management features for your challenge, you can also view assessment information directly on the challenge’s project board.
To do so, navigate to the project board screen for your challenge. As long as the appropriate ideas have been added to the board, then their aggregate scores can be viewed directly on the card.
Clicking the idea card brings up the idea’s project screen. From here the detailed per-assessor and per-section breakdown can also be viewed.
7.3. Download an assessment report for your challenge
SimplyDo also allows you to download a report containing the assessment information for your challenge. This report is an Excel spreadsheet that contains a number of sheets with various pieces of information included.
To download the report navigate to the reports tab of your challenge’s settings screen. From this tab, click the new report button to download your report.
You can also choose whether to include assessor notes in the downloaded report. To do so, simply check the display assessor notes in the report checkbox before downloading the report.
In this article we have outlined the process for managing assessments in your challenges.
If you want to start using assessments, or have any questions or problems with this setup, please just get in touch with your account manager.