Much of the literature on literacy assessment is grounded in English literacy (or more broadly, literacy in alphabetic languages). However the approach to measurement and evaluation may need to be adpated depending on the mother tongue of the learners, language of instruction, or in consideration of multilingual contexts. Other contextual factors-- such as location (urban vs. rural), proportion of out of school children, community literacy rates-- may impact the measurement approach used, and the corresponding level of effort required for carrying out the activity.

Census approaches ensure that data accurately reflect a population, since most or all of the research subjects are included in data collection, however they may be cost- and time-prohibitive. Sampling approaches that ensure representativeness can produce results that are just as reliable. Constructing a representative sample does require forethought, planning, and some understanding of statistcs. The resources included in this section provide some information and considerations with regards to sampling.

It is not uncommon for hard-to-reach students to be left out of measurement activities.  These may be out of school children, who are hard to access because they don't attend school, individuals with disabilities for whom data collection tools might need to be adapted, and students who do not speak the language of instruction or assessment. When these individuals are left out of assessment activities, the status of their learning is left unknown.  While additional effort may be necessary for making measurement an inclusive activity, it is important for ensuring a complete measurement activity that provides the full picture of learning within a community or institution.

All assessment tools should be checked by domain and assessment experts to ensure that the test has sufficient items, addressing an appropriate range of content, contexts and skills that are suited to the test takers’ levels of skill, to meet the claim that it assesses a particular domain. For example, a valid reading comprehension test should include a range of comprehension skills, different texts and at least 20 items that require reading the text(s) and cannot be answered correctly by guessing or general knowledge. Quality assessments include documentation and evidence of validity, with a clear definition of the construct and scope of the test. Pilot testing and analysis can check if items are assessing skills related to the relevant domain.

Assessments need to be reliable, so that all students have the same opportunity to demonstrate their skill and are not disadvantaged through unclear instructions, variable administrative procedures, bias in scoring responses or other factors that are not related to the skills being assessed. Administration and scoring procedures need to very clear in order to be consistent.  Training is generally essential. Pilot testing will also allow you to undertake quality assurance checks to ensure that tests are reliable.  Different aspects of reliability can be evaluated such as internal consistency, or inter-rater reliability, using a range of metrics.

Numbers only tell us part of the story. Contextualizing your results and making them understandable to key stakeholders is important for operationalizing and learning from your measurement exercise.  Moreover, thinking through the use of your data before you collect it will help ensure its relevance, and ensure that the results of your measurement exercise are used, rather than sitting around collecting dust.

Often times community engagement stops at the programmatic level, however there are ample opportunities to include community members in the measurement process. Engaging communities in assessment and measurement activities can help raise awareness around learning levels and help ensure that underrepresented groups are included in assessment and measurement activities.