Big shadow test Big-Shadow-Test Method is used to solve a large simultaneous problem as a sequence of smaller simultaneous problems. Shadow tests are no regular tests; their items are always returned to the pool. They are only assembled to balance the selection of items between current and future tests. Because of their presence, they neutralize the greedy character inherent in sequential test-assembly methods. In doing so, they prevent the best items from being assigned only to earlier tests and keep the later test-assembly problems feasible.

1. BIG-SHADOW-TEST
METHOD
Presented to
Dr. Nasir Mahmood
Presented by Muhammad Munsif
munsifsail@gmial.com
Mphil Education(Evening)

2. BIG-SHADOW-TEST METHOD
• Big-Shadow-Test Method is used to solve a large simultaneous
problem as a sequence of smaller simultaneous problems.
• Shadow tests are no regular tests; their items are always returned to
the pool. They are only assembled to balance the selection of items
between current and future tests. Because of their presence, they
neutralize the greedy character inherent in sequential test-assembly
methods. In doing so, they prevent the best items from being
assigned only to earlier tests and keep the later test-assembly
problems feasible.

3. Methodology
• Models for the assembly of multiple tests lead to problems larger
than those for single tests. The number of decision variables are
necessary to formulate a problem of T tests from a pool of I items is
equal to TI. This number increases linearly with the number of tests.
• Models for multiple tests also always have I more constraints than
those for single tests because of the no-overlap constraints. If we
want to control the overlap between pairs of tests, the increase
becomes much larger.

4. Methodology
• In the worst case, with overlap controlled between each pair of tests, the
number of variables is equal to TI +(
𝑇
2
)I and the model has (
𝑇
2
)I more
constraints to specify the required overlap, where (
𝑇
2
) is the binomial
coefficient.
• Due to recent optimization of commercial MIP solvers, problem size is no
longer the limiting factor it used to be. Nevertheless, it may be convenient
to have an alternative method for problems that still appear to be too
large. A useful backup method is the big-shadow-test method explained in
this section.

5. Graphical Presentation

6. Advantages
• A shadow test is a special case of content-constrained CAT (computerized adaptive
testing) that explicitly uses ATA (automated test assembly) for each adaptive item
selection.
• This model blends the efficiency of CAT with the difficulty of using powerful linear
programming techniques (or other ATA heuristics) to ensure a psychometrically
optimal test that simultaneously meets any number of test-level specifications and
item attribute constraints.
• Shadow testing can further incorporate exposure control mechanisms as a security
measure to combat some types of cheating (van der Linden, 2000, 2010).
• It does not require simulation studies to establish the item exposure parameters for
the items before administering a test.

7. Disadvantages
• Shadow testing is a mathematically elegant model for CAT that has not been
implemented to date in a real CBT (computer based testing) system.
• Simulation research conducted with paper-and-pencil item banks from the Law
School Admissions Test shows extreme promise (van der Linden & Reese, 1998)
but is hardly conclusive.
• A predictable complication with shadow testing that relates directly to system
performance, especially with regard to Web-based testing (WBT).
• Shadow testing requires that a powerful linear programming software package
be fully integrated as part of the test-delivery software driver (Diao & van der
Linden, 2011).

8. Disadvantages
• Commercial linear programming software packages do exist (e.g., the CPLEX
Optimization Studio available from IBM), they will be costly and complicated to
integrate with the current class of test-delivery applications available throughout
most of CBT world.
• Furthermore, even if implemented, the impact on system performance is
unknown for WBT (or large-network installations) running most of the required
computations and data management routines on the server side. Unless these
pragmatic systems issues can be resolved and allow content-constrained CAT
with shadow testing to gain widespread use, it may remain an elegant (and
somewhat costly) solution that remains “on the shelf.”

9. Limitations
• Maximizes information while handling content and
other constraints efficiently in real time, using linear
programming optimization.

10. Conclusion
• The big-shadow-test method is a general heuristic scheme. It has four features that
distinguish it from the more specialized heuristics we reviewed earlier.
• First, the degree to which the method behaves as a heuristic can be controlled by the
test assembler. The critical parameter is the number of steps. The model, with one
single test at each step, has T−1 steps and illustrates one extreme of the range of
possibilities. The simultaneous model, with T tests and no shadow test, is the other
extreme. The smaller the number of steps, the closer the result can be expected to be to
the exact solution obtained by the simultaneous model. The only restriction in our
attempt to get as close as possible to the exact solution is computation time.

11. Conclusion
• Second, unlike the heuristics with second-stage item swapping, the big-shadow-
test method looks ahead and prevents unbalanced solutions instead of fixing them
after the fact.
• Third, whereas other heuristics are typically formulated for a specific type of
objective function and/or class of constraints, the big-shadow-test method is based
on a general scheme that can be used with any type of problem for which a model
for a single test can be formulated; that is, with any of the models.
• Finally, as already indicated in the model, the big-shadow-test method enables us
to assemble a set of tests for relative targets whose heights are maximized
simultaneously. This feature cannot be realized by a purely sequential heuristic.