More thoughts on CBT exams

As of right now, we are about 6 weeks out from the fall 2020 exam date. I was hoping that by this time we would have some sample questions from the SOA to use as a guide for what to expect under the new format. I’m sure they are working hard on developing additional guidance, and it’s taking a major effort involving many people—both paid and unpaid—in a year when many people are constrained due to a variety of challenges in their personal lives as well.

In addition to all the Excel-based exercises we have posted in certain ILA courses, I want you to know that at a minimum we will post a reformatted version of our existing practice exam some time in the month of October. We are working on converting our existing practice exams to a Word/Excel format and also trying to develop “model solutions” that mimic best practices and techniques that we think will be optimal under exam conditions.

One thing that has surprised me as I create Word- and Excel-based solutions, even from existing known solutions: I’m really not that much faster overall than I was with pen and paper. Even though some things are lightning fast in Word and Excel (e.g. copy/paste/fill operations), a number of things are very time-consuming. By the time I finish my calculations and properly label my work in Excel, I’ve spent close to the same amount of time I would if I had done the same problem by hand.

With that in mind, it’s possible that the first rendition of the fall 2020 CBT exams will be based on problems very similar in size/scope to conventional written exam problems. This is more or less the approach that the CAS is taking in their move to CBT. They are planning to basically test the same type and number of problems, but change the answer format to Excel-only. The SOA has stated that candidates will have both a Word file and Excel file, but how the files will be used for answers may vary from exam-to-exam. That’s all we know right now.

I am planning to develop more videos and guidance for CBT, but I have been waiting for a number of little details like the above to emerge. I don’t want to start giving very specific advice based on speculation. However, here is a list of thoughts to keep in mind based on where I am in my thought process. I’ve also covered some of these tips in our new Excel-based exercises posted in some ILA courses:

  • Don’t automate in Excel!
    • Take the easiest, fastest, most transparent approach 
    • Avoid complex date functions, etc.
  • Try to avoid really long nested Excel formulas
    • Use more cells and label steps
    • Split big formulas into separate cells, rows, columns, etc. so you can label the pieces and also clearly see the inputs. This makes it much easier to spot mistakes, which can easily be hidden inside large formulas.
  • Label work liberally—ideally to the right of the cell where you do calculations
    • I really like this approach for efficiency and transparency. It’s tempting to try to make things look nice and pretty in a table, but my brain in Excel likes to do a calculation first and then describe it second. By writing the label to the right, I don’t have to worry about the text overflowing the cell. I can be more verbose. 
    • Whatever you do, be sure to describe your work/formulas/intentions somewhere in in Word and/or Excel before, during, or after calculations
    • Excel is powerful but it’s just as easy to trip over yourself and make mistakes, which can often be hidden under the surface of the values.
  • When it comes to writing formulas, I think it’s easier to describe them verbally instead of doing fancy notation
    • It’s faster to write “PV of $1 over 10 years discounted with survivorship and 3% interest” than it is to write the proper formula for an annuity due, complete with dots and subscripts.
    • The equation editor in Word is decent and also supports most basic LaTeX-style syntax, but I don’t think it’s reasonable to expect every candidate to be proficient in that.
  • When studying, don’t get over-reliant on advanced functionality in the version of Word and Excel that you have
    • We have no idea how “feature complete” the versions of Word/Excel in the Prometric environment will be. For example, on some CPA exams given in Prometric, various keyboard shortcuts and even the F keys are disabled. 
    • Don’t count on add-ins and special custom functions to be available
  • In Word, when answering verbal questions, write out high-point-value bullets first, then fill in detail
  • Always think in terms of auditability — leave a clear trail in Excel