Tagged: review feedback
November 23, 2017 at 3:45 pm #429
Had planned to upload this as a PDF, but don’t see any upload mechanism. So here’s a copy-and-paste which will hopefully retain enough of the formatting to be clear.
TEST NOTES 2017/11/22
CAN I RETIRE YET RETIREMENT APP (PRO 1.0.1)
N.B. Updated to 1.0.2 on 2017/11/23 but it doesn’t appear to have any differences affecting these comments.
HYPOTHETICAL USE CASE
Married couple M=52 (“You”), F=54 (“Spouse”); filing jointly with two dependent children. One child is in college; the other is a junior in high school. Education is paid for from fully funded 529 plans; neither these accounts nor the bulk of expenses were included in the use case.
M is still working ($120,000/year pre-tax) and hopes to retire at age 55. F retired at age 45.
They have a net worth of $4 million. Asset allocation is 70/20/10 stocks, bonds and cash. They rebalance annually.
• Tax free (Roths) = $0.5 M
• Tax deferred = $1.5 M (M), $1M (F)
• Taxable = $1 M
They plan to take social security at age 70. They estimate (conservatively) that annual SS will be $30,000 and $17,000 (F) respectively.
With the above information, it was extremely easy to enter in the personal information into the app. Life expectancy was given as 92 (M) and 95 (F). I found the placement of “Asset Classes” above “Accounts” and “Reports” initially confusing. People, Accounts and Events are information about the couple or person. The “Asset Classes” is about expected returns that you set (currently fixed) just prior to running the simulation. I suggest moving Asset Classes to just above Reports and calling it Asset Class Returns.
The Events section has a lot of different categories and I did not explore them all. I originally missed the “Annual Living Expense” since I thought of income as more significant than an “Event”. It asks you how often you want to withdraw (Amount Type) and from what account (From Account). I found this confusing since one would withdraw from all accounts eventually. Is it asking which account to start withdrawals?
I ran this Use Case on several other retirement calculators such as FireCALC, AnalyzeNow, CrowdSourceFIRE (cFIRESim), ORP, etc. Some of these are Monte Carlo some use historical data. The income for a 95% survival rate and 40-year period ranges from $140,000 to $200,000/year. I used a Living Expense somewhere near the median of these values.
Reports. The tool is what is called a deterministic calculator, where the rates of return are fixed and it produces reports and plots of net worth and income with time. Quicken and Vanguard have similar deterministic calculators. Here is where my biggest confusion lies with this App. What are these reports trying to tell me? Is it like ORP, giving advice on what accounts to tap as a function of time? Can I use it to (roughly) estimate the maximum withdrawal rate? I suspect the report has a lot of information in it but I am unable to figure out how it is calculating the columns. By running enough simulations, I could reverse engineer this eventually but I’ve run out of time. Perhaps some additional documentation is needed to explain the calculation process?
GENERAL COMMENTS (FROM TWO USERS)
• I found the app very easy to use and the interface was mostly intuitive. With the exception of the Asset Classes and the Annual Living Expenses (see above), everything appeared to be where it should be.
• The app UI doesn’t quite follow the Android “standard”. E.g. when in a particular function screen, there is no “back” arrow at upper left to return to the previous screen. (If you prefer not to implement this, perhaps tapping on the app icon could be used as a “home” (top-level screen) shortcut, rather than using the phone’s own “Back” button to step back through levels.)
• Similarly, a small “Done” and/or “Cancel” button (or checkmark and X) at the top of data-entry screens would be both useful and more Android-standard. Changes appear to be effective immediately, so to change your mind about changes :), you have to remember what the original values were and put them back.
• On the screens for different subcategories such as Events, the top title displays, for example, “Events/Property”. Since “/” is often used to mean “or”, I suggest using a colon there instead. Unlike the slash, this is instantly intuitive to parse (and would be familiar to anyone who has used Quicken expense subcategories). Takes up a tiny bit less title-bar real estate, too.
• When entering information into a screen, the tool nicely highlights with a red line. However, once you have reached the last line and enter Done, the line stays red. The first time this happened I thought I had made an error on that line. Maybe after you tap Done, the red line could go away? [Note from other tester more familiar with Android: the “Done” button being referred to here is the one on the text-entry keyboard. It does not actually exit the screen; hence the red line stays red to show which field is currently selected. See suggestion above about adding Done/Cancel options for data entry screens in the CIRY app itself.]
• When entering information onto a screen with many fields, the text-entry keyboard obscures the lower area of the fields. Although the “Next” button on the keyboard allows you to advance to the next field, you may have to tap it several times to get to the one you want. Could the list be finger-scrollable in this situation? It allows that sometimes, but it’s inconsistent. It isn’t strictly related to whether all the fields fit on one screen initially, and in some cases when it does let you scroll, it doesn’t seem to recognize that some fields are still hidden by the keyboard, so “Next” is the only way to reach them (or exit the keyboard and tap on them directly). Other times full scrolling works fine.
• My old eyes like to see commas between all those zeros. It is easy to lose a zero typing $1000000, while $1,000,000 is nice and clear.
• If you tap on the question mark (?) at the top right corner you are always taken to the start of the help manual. It would help if there were pointers from subsections of the tool to the help file for that subsection. For example, if you are in Accounts and tap the “?” it should take you to the help file section that describes the entries in Accounts.
• When defining a new Event, it should be possible to enter the name at creation time, as is done when defining a new Asset Class. At the moment, you have to pick from a list – as on the new Asset Class drop-down – but then it’s created and added to the Event screen immediately. You have to long-press to rename, and separately tap to enter parameters. So it starts off looking like it will work the same way as a new Asset Class and then does something very different.
[DETAILED LOGS FROM SEVERAL RUNS ARE AVAILABLE TO DEVELOPERS PRIVATELY ON REQUEST]
• During two days of fairly intensive testing, the app crashed (“Unfortunately, <CIRY> has stopped”) four times. Restarting the app gave no trouble. We did not select “Report” for these, but can do that another time if you would like. (Where does it go?)
• The only way to change the returns on asset classes is to define your own new ones. This is described in the video, but not in the in-app help (it should be). It would be more intuitive if they could just be edited directly, with a “reset to default” button.
• Once you’ve defined your own asset class, you can’t edit its settings, either. This makes it very cumbersome to tweak returns to see how they affect the calculation results. This is an important aspect to consider for users of retirement calculators, e.g. if you want to try more/less conservative asset return scenarios. Please allow direct editing of these values, at least on the user-defined classes.
• As far as we can tell, the Account types don’t directly allow for the possibility that individuals may have more than one account of the same type (e.g. tax-deferred) which may have significantly different asset mixtures and in some cases different rules such as penalty-free starting age. If you want a reasonably accurate calculation, it’s necessary to make some assumptions about relative value weighting of the two accounts and then manually estimate the combined return. Rebalancing strategies may also be affected by this artificial rearrangement. Suggestion: allow the user to create more than one account within each type.
• The “Living Expenses” event should clarify whether the amount you enter is intended to include taxes. Right now it doesn’t state one way or another; only the detailed log on the results shows they are not included. There is nothing in the in-app help about this or other assumptions that may be made in the predefined events, including the “defaultPro” scenario in the overall app Settings.
• State income tax is never defined. The detailed log shows that it is calculated as 0%, which will not be true for most U.S. residents. This setting is undocumented as far as I can tell, but the impact in some states is high. Rather than leaving it for the user to figure out that it isn’t covered – which they never will if they don’t carefully examine the detailed log – I suggest including a field to define a number someplace obvious – e.g. in the “People” section – to ask for “State Tax Est %” and use that. (If you instead asked for something like “State of Residence”, the app would need to look up the tax in a table, which would be a maintenance headache for you.)
• The drawdown timeline drains accounts strictly in the order they’re defined in the Accounts list. In our test case, the Spouse is older than You. The taxable account runs out before You reaches eligibility age for most types of tax-deferred withdrawals; however, Spouse has reached it by then. The reports show that the “You” tax-deferred account is tapped first, even though doing so incurs an early-withdrawal penalty and there would not have been one if the Spouse tax-deferred account were used instead. In real life one would never create such a situation unnecessarily. The app should check whether a penalty can be avoided by using a different account first. Alternatively, clearly indicate that “You” should be set up to be the oldest person in a couple.
• The dollar amount of the early-withdrawal penalty does not seem to be stated in the detailed results log.
• In the results log, the “GROWTH—–“ section heading is always appended to the previous line. It should start on a new line, as the other section headings do.
• The results log sometimes shows “TAXES / Spouse, You / filing status: SINGLE” at the beginning, but in the rest of the log it states “MARRIED_FILING_JOINTLY”. On other runs it shows the correct status in all places.
• Similarly, the results log sometimes shows “today’s dollars: false” even though the app settings have selected Today’s Dollars. Other times it is shown correctly. I *think* it is doing the right thing despite this.
• All RMD in excess of defined expenses is swept into Cash. As an enhancement, you might ask the user what percentage of any excess they would like to put into the taxable investment account instead. This could make a difference to the long-term outcome, possibly a substantial one.
November 23, 2017 at 7:17 pm #430
Thanks so much for the detailed testing and feedback nmgyrl! This represents a lot of effort, dedication, and thought on your part, and we appreciate it. Glad the app was easy to use and mostly intuitive.
I’ll try to reply to most of your other points now, but will need to let a few sit for later:
Asset classes are required input for setting the asset allocation in accounts. So it seems to us that the asset class section belongs before accounts. But we’ll see what other users think about that…
The Pro calculator is a general purpose modeling tool. We seed it with some sample events, but there are many different ways that somebody could model their situation. However, depending on what kind of users we get, we may need to do more in the way of providing built-in structure.
Yes, the account specified for events such as the Withdrawal event is the account to *start* taking withdrawals from. Once that account balance is zero, the program works its way up the built-in account sequence.
The model produces a high-fidelity view of your cash flow in retirement. It is currently up to the user to then optimize that cash flow to their purposes. Over time we may add some automation to that process, but this is the initial release of a very inexpensive app so expectations should be modest.
The Cash Flow report would benefit from more columns with intermediate values. This is high on our task list after v1.0
We did our best to follow Android standards when they were clear. But they were surprisingly unclear (and buggy) in a number of scenarios. I’ve logged several of your comments in our tracking system for consideration in evolving our user interface.
Unfortunately, we sweated bullets to save everything immediately, without OK/Cancel, which we didn’t perceive to be the Android standard. We’ll see what other users say now. 🙂
As far as I know, we don’t have much control over that focus red line, the edit keyboard, or underlying scrolling. I believe those are Android system services. But my co-developer may be able to expand on this.
I miss the commas too. I believe they were a validation headache in data entry, but let’s see how much other users want them.
True context-sensitive help is high on our task list. The help system is currently formatted for it, but the program just isn’t jumping into the specific subsections yet.
Good suggestion on entering event names at creation time. It’s been added to our list!
We have not seen any repeatable app crashes here. Obviously these are bad and we’d like to get to the bottom of them. Reports should come straight to our support address, but my co-developer will need to confirm that. At this stage, we’d be most interested in the steps to duplicate, if possible.
You are entirely right that editing asset class returns directly would be highly desirable. The current design has usability implications that we don’t like, but it just wasn’t possible to change late in the game. It is high on the list for a future release. Meanwhile it is possible to change account returns fairly quickly by setting up alternative asset classes and swapping them into the asset allocations.
We have the capability internally for users to create any number of their own arbitrary account types. But for simplicity and because this is an inexpensive app, it’s one area we haven’t exposed yet. We may add it in a future release, or make it an add-on feature.
We will keep working on documentation. We are already drifting into giving modeling/financial advice and we’ll have to decide how far we can afford to go with that. It is generally true, since the program includes sophisticated tax calculations, that users don’t need to consider taxes when entering events.
We already handle a state/local effective tax rate internally, but it simply isn’t exposed in the user interface. We’ll make that a higher priority.
User-defined account withdrawal order is another feature that is available internally but not yet exposed in the user interface. You make a good case for why it needs to be there sooner, than later.
There is plenty of work we could do to improve the Log. It started life as internal debugging tool, but hopefully will be an increasingly useful report to users. Thanks for the detailed suggestions. All placed on our task list.
Unless and until we provide automated cash flow optimization (again this is currently an inexpensive app), the program’s events provide significant power for personal customization. For example, Transfer events, using the Percent Amount Type, should allow for some control over investing excess RMDs from the Cash account.
November 25, 2017 at 2:54 pm #431
Yes, thank you for taking the time to give us your detailed feedback, it is very helpful, nmgyrl.
Separate to Darrow’s comments, let me say that getting the right balance in the UI has been an ongoing effort throughout development and with feedback, we will be making adjustments.
The focus red line behavior is the default, we will investigate better visual indication that entry is complete. We had the commas in the entry field for months and took them out, as the default behavior required you to “get them right” … we will rework it.
Crash reports from the Google Play Console are only collected if the user has opted in to automatically Share usage & diagnostics information with Google, which I understand not everyone will do. The setting is system wide, rather than per application and I have no crash reports sofar. If you can outline the sequence of steps that led to the crash, I would certainly investigate.
You must be logged in to reply to this topic.