62. EPMO Open Source Coordination Office Redaction File Detail Report

Produced by Araxis Merge on 11/8/2016 5:25:02 PM Eastern Standard Time. See www.araxis.com for information about Merge. This report uses XHTML and CSS2, and is best viewed with a modern standards-compliant browser. For optimum results when printing this report, use landscape orientation and enable printing of background images and colours in your browser.

62.1 Files compared

#	Location	File	Last Modified
1	rxrefill-v2.0.0 P2.zip	MAP2+Master+Test+Plan_Program-Level.docx	Fri Nov 4 20:30:29 2016 UTC
2	rxrefill-v2.0.0 P2.zip	MAP2+Master+Test+Plan_Program-Level.docx	Sun Nov 6 22:40:39 2016 UTC

62.2 Comparison summary

Description	Between Files 1 and 2
Description	Text Blocks	Lines
Unchanged	6	1758
Changed	5	10
Inserted	0	0
Removed	0	0

62.3 Comparison options

Whitespace
Character case	Differences in character case are significant
Line endings	Differences in line endings (`CR` and `LF` characters) are ignored
CR/LF characters	Not shown in the comparison detail

62.4 Active regular expressions

No regular expressions were active.

62.5 Comparison detail

	1	MAP2 Maste r Test Pla n_Program- Level
	2	Introducti on
	3	Department of Vetera ns Affairs (VA) Conn ected Heal th Office (CHO) is s eeking to benefit fr om advance s in mobil e technolo gy in orde r to:
	4	Improve th e quality of health of Veteran s
	5	Increase t he quality of health care avail able throu gh the VHA
	6	Improve th e efficien cy of prov iders as w ell as sup porting st aff
	7	Continuous ly expand Veterans' overall sa tisfaction with the Department of Vetera ns Affairs (VA)
	8	To this en d, the VHA CHO has a warded Hew lett-Packa rd Enterpr ise Servic es (HPES) a contract for the M obile Appl ications ( Apps) Phas e Two (MAP 2) program to provid e Agile mo bile appli cation dev elopment a nd enhance ment suppo rt includi ng program managemen t, applica tion initi ation, dev elopment, and testin g support for ten ap plications supportin g both the VA caregi ver commun ity and th e external Veteran c ommunity. This contr act also i ncludes su pport for compliance testing, field test ing, and r elease sup port and a pplication deploymen t.The MAP2 program-l evel Maste r Test Pla n (MTP) wi ll guide a ll stakeho lders thro ugh the te sting cycl e of any a pplication developed or enhanc ed by the MAP2 progr am team. E ach sectio n will sta te, at a h igh level, the 'what and why' of testing and give the manage rs one pla ce to go t o understa nd the tes ting appro ach used b y the MAP2 Test team .
	9	Purpose
	10	The purpos e of this program-le vel MTP fo r the MAP2 program i s to defin e all test levels th at must be performed on MAP2 m obile appl ications b efore the applicatio n can be d eemed read y for subm ission to the VA Com pliance Re view and V erificatio n and Vali dation tes ting proce sses. A se parate MTP will be d eveloped f or each ne w or enhan ced mobile applicati on.
	11	Test Objec tives
	12	This progr am-level M TP support s the foll owing obje ctives:
	13	To provide test cove rage for 1 00% of the documente d requirem ents – fun ctional an d non-func tional con tained in applicable requireme nt documen tation (i. e., the Bu siness Req uirements Documents (BRD), the Requireme nt Specifi cations Do cument (RS D), and Us er Stories )
	14	To validat e the User Story Acc eptance Cr iteria of each mobil e applicat ion has be en met
	15	To provide coverage for applic ation Syst em Design Document ( SDD) eleme nts
	16	To provide a baselin e of neede d function ality of t he Mobile Applicatio n Environm ent (MAE) test envir onments
	17	To perform handoffs with Verif ication an d Validati on (V&V), business s ponsors, a nd Complia nce bodies for indep endent tes ting, comp liance rev iews, and user funct ionality t esting
	18	Function a s a refere nce docume nt for mob ile applic ation deve lopers on testing st andards to be applie d to mobil e applicat ions enter ing the MA E
	19	Acronyms
	20	This secti on contain s a list o f acronyms used in t his docume nt.
	21	Acronym
	22	Descriptio n
	23	ADM
	24	Agile Deve lopment Me thodology
	25	AAP
	26	Ask a Phar macist
	27	A-CHESS
	28	Addiction Comprehens ive Health Enhanceme nt Support System
	29	ADR
	30	Administra tive Data Repository
	31	BRD
	32	Business R equirement s Document
	33	CCB
	34	Change Con trol Board
	35	CDW
	36	Corporate Data Wareh ouse
	37	COR
	38	Contractin g Officer' s Represen tative
	39	EAS
	40	External A ssessment Services
	41	ESE
	42	Enterprise Systems E ngineering
	43	GRECC
	44	Geriatric Research E ducation a nd Clinica l Center
	45	HPES
	46	Hewlett-Pa ckard Ente rprise Ser vices
	47	IAM
	48	Identity a nd Access Management
	49	ID
	50	Identifica tion
	51	IOC
	52	Initial Op erating Ca pability
	53	MA
	54	Mobile App lication
	55	MADW
	56	Mobile App lication D evelopment Workflows
	57	MAE
	58	Mobile App lication E nvironment
	59	MDWS
	60	Medical Do main Web S ervices
	61	MHV
	62	MyHealtheV et
	63	MTP
	64	Master Tes t Plan
	65	MVI
	66	Master Vet eran Index
	67	MHPRO
	68	Mental Hea lth Patien t Reported Outcome
	69	NFR
	70	Non-Functi onal Requi rement
	71	NIST
	72	National I nstitute o f Standard s and Tech nology
	73	NSOC
	74	Network Se curity Ope rations Ce nter
	75	OIA
	76	Office of Informatio n Assuranc e or Offic e of Infor matics and Analytics
	77	OI&T
	78	Office of Informatio n and Tech nology
	79	O&M
	80	Operations and Maint enance
	81	PFNMN
	82	Post Falls Note Mobi le Nursing
	83	PHI
	84	Personal H ealth Iden tifiers
	85	PII
	86	Personally Identifia ble Inform ation
	87	PMAS
	88	Project Ma nagement a nd Account ability Sy stem
	89	RSD
	90	Requiremen ts Specifi cation Doc ument
	91	RTM
	92	Requiremen ts Traceab ility Matr ix
	93	SDD
	94	System Des ign Docume nt
	95	SQA
	96	Software Q uality Ass urance
	97	SSP
	98	System Sec urity Plan
	99	UFT
	100	User Funct ionality T est
	101	URL
	102	Uniform Re source Loc ator
	103	VA
	104	Department of Vetera ns Adminis tration
	105	VA-GDx
	106	VA Genetic Diagnosti c Testing
	107	VAHA
	108	VA Health Adapter
	109	VAMF
	110	VA Mobile Framework
	111	VDD
	112	Version De scription Document
	113	VHA
	114	Veterans H ealth Admi nistration
	115	VistA
	116	Veterans H ealth Info rmation Sy stems and Technology Architect ure
	117	V&V
	118	Verificati on and Val idation
	119	Roles and Responsibi lities
	120	Table 1 li sts the ke y roles an d their re sponsibili ties for t his Master Test Plan . The role s defined are member s of the H PES MAP2 P rogram tea m. Table 1 : Roles an d Responsi bilities
	121	Role
	122	Descriptio n
	123	Developers
	124	Persons th at build o r construc t the mobi le applica tion compo nents. Dev elopers wi ll be resp onsible fo r unit and component -integrati on testing .
	125	Developmen t Manager
	126	Person res ponsible f or ensurin g mobile a pplication component s have gon e through adequate u nit and co mponent-in tegration testing by developme nt team be fore turni ng over to the Syste m Quality Assurance (SQA) Test Team for system tes ting. Resp onsible fo r approvin g the MTP.
	127	Project Ma nager
	128	Person res ponsible f or oversee ing the de velopment and testin g of mobil e applicat ions under the MAP2 program. R esponsible for appro ving the M TP.
	129	Program Ma nager
	130	Person tha t has over all respon sibility f or the suc cessful pl anning and execution of the MA P2 program . The Prog ram Manage r will be responsibl e for appr oving the MTP.
	131	Stakeholde rs
	132	Persons th at hold a stake in a situation in which they may a ffect or b e affected by the ou tcome.
	133	SQA Test T eam
	134	Persons re sponsible for ensuri ng full ex ecution of the test process (i .e., test case devel opment, te st case ex ecution, e tc.) for e ach mobile applicati on.
	135	SQA Test P lanner/Eng ineer
	136	An experie nced test analyst or member of the Test Team that leads and coordinate s activiti es (i.e., ensure tes t environm ent setup, support I nitial Ope rating Cap ability (I OC) and V& V testing activities , etc.) re lated to a ll aspects of testin g for an i ndividual mobile app lication. Responsibl e for deve loping a M TP for eac h applicat ion.
	137	Test Envir onment Tea m
	138	Persons th at establi sh, mainta in, and co ntrol test environme nts.
	139	SQA Test M anager
	140	Person tha t has over all respon sibility f or develop ing the pr ogram-leve l MTP. Per son respon sible for leading an d coordina ting activ ities rela ted to all aspects o f testing based on t he approve d program- level Mast er Test Pl an and app lication–l evel MTPs and schedu les.
	141	Processes and Refere nces
	142	The proces ses that g uide the i mplementat ion of thi s Master T est Plan a re:
	143	VA's Mobil e Applicat ion Develo pment Life Cycle
	144	VA's Mobil e Applicat ion Projec t Manageme nt Account ability Sy stem
	145	MADW-3 App lication D evelopment document
	146	MADW-4 Ver ification and Valida tion
	147	MADW-5 Com pliance Re view
	148	MADW-6 Use r Acceptan ce Test
	149	MADW-7 Fie ld Test (I OC)
	150	The refere nces that support th e implemen tation of this Maste r Test Pla n are:
	151	Mobile Dev elopment A pplication Life Cycl e (http:/ DNS /content/d eveloping- va-apps)
	152	ProPath
	153	(http:// DNS /process/p ropath)
	154	Section 50 8 Office W eb Page
	155	(http:// DNS /508workgr oup)
	156	Privacy Im pact Asses sment - Pr ivacy Serv ice
	157	(http:// DNS /Privacy_I mpact_Asse ssment.asp ) Note: Fo r 508 Comp liance, co py and pas te the abo ve URLs in to your br owser to r each each site. Also , the abov e hyperlin ks/URLs pr efaced wit h DNS are only accessible with acce ss to VA n etwork.
	158	Items To B e Tested
	159	Note: Addi tional inf ormation f or the Ite ms to Be T est sectio n will be added/upda ted in fut ure iterat ions, as n eeded.This section l ists all t est items (i.e., fun ctional or features and non-fu nctional r equirement s derived from the B RD, RSD, U ser Story documentat ion, etc.) and inter faces and/ or service s (i.e., d erived fro m the RSD addendum, SDD addend um, etc.) to be test ed for a n ew or enha nced mobil e applicat ion as par t of the M AP2 progra m. After d elivery an d review o f the SDD addendum, the HPES S QA Test te am will de termine th e kind of integratio n testing that needs to occur for a give n applicat ion in add ition to t he other s ystem test ing to occ ur on the mobile app lication. The follow ing compon ents and f eatures (a nd combina tions of c omponents and featur es) will b e tested:
	160	1. Applic ation Sour ce Code An alysis (HP ES Fortify Scan tool ).
	161	2. Mobil e applicat ion testin g (functio nal and no n-function al):
	162	Rx Refill Delivery T racking an d Image En hancements
	163	MyHealtheV et (MHV) S ervice Con version
	164	VA PTSD Co ach Phase Two (2)
	165	Journal Ph ase Two (2 )
	166	Ask a Phar macist (AA P) – (New Applicatio n)
	167	Mental Hea lth Patien t Reported Outcome ( MHPRO) – ( New Applic ation)
	168	VA Genetic Diagnosti c Testing (VA-GDx) – (New Appl ication)
	169	Addiction Comprehens ive Health Enhanceme nt Support System (A -CHESS) – (New Appli cation)
	170	Geriatric Research E ducation a nd Clinica l Center ( GRECC) – ( New Applic ation)
	171	Post Falls Note Mobi le Nursing – (New Ap plication)
	172	3. Mobile Adapter te sting.
	173	4. Documen tation Rev iews.
	174	5. Middlew are Update s and Addi tions (i.e ., VA Mobi le Framewo rk (VAMF), VA Health Adapter).
	175	Refer to S ection 3 o f this doc ument to s ee the com pliance bo dies invol ved in the mobile ap plication reviews ba sed on the risk leve l that the mobile ap plications present.
	176	Applicatio n Source C ode Analys is
	177	The MAP2 D evelopment team will perform a static co de analysi s during t he applica tion devel opment sta ge. The HP ES Fortify tool will be used f or this an alysis. Ou tput from the analys is will pr ovide the MAP2 HPES Developmen t team wit h immediat e feedback regarding security vulnerabil ities and remediate these issu es before sending th e applicat ion on for further r eview by o ther compl iance revi ew bodies.
	178	The MAP2 HPES Devel opment tea m will exe cute the H PES Fortif y tool aga inst any c hanged or added serv ices that were creat ed to supp ort the ne w or enhan ced mobile applicati ons as par t of the M AP2 progra m. The cri tical and high error s will be fixed prio r to turni ng the cod e over for VA compli ance revie ws.
	179	Mobile App lication T esting
	180	Each mobil e applicat ion will i nclude a M aster Test Plan (MTP ), which w ill list t he testing specific to that mo bile appli cation tha t is perfo rmed by th e MAP2 Tes t team. Th e MAP2 Tes t team inc ludes both HPES Deve lopers and SQA Teste rs working within a Scrum team supportin g the Agil e Developm ent Method ology (ADM ) that is being foll owed by th e MAP2 pro gram team. The MTP i ncludes th e Test Inc lusions (t he feature s or funct ions to be tested), non-functi onal requi rements, a nd the det ailed test cases and test scri pts. It is expected that devel opers on t he Scrum T eam are ex ecuting th e Product Component Tests (uni t-level) a nd Compone nt-Integra tion Tests for the m obile appl ication.
	181	Next, the HPES SQA t ester assi gned to th e Scrum te am will pe rform feat ure verifi cation tes ting durin g each spr int iterat ion, which involves testing ag ainst the user story 's accepta nce criter ia. Both a utomated a nd manual test cases /scripts w ill be use d during t his phase of testing .
	182	Mobile Ada pter Testi ng
	183	If a MAP2 mobile app lication r equires a new Mobile Adapter s ervice or a revision to the se rvice, the MAP2 HPES Developme nt team mu st test th e changes that affec t the Mobi le Adapter . The MAP2 HPES Deve lopment Te am must pe rform a Pr oduct Comp onent and Component- Integratio n Test for the updat ed or new service. T he System Tests will be perfor med by the MAP2 HPES SQA Test team.
	184	Documentat ion Review s
	185	Prior to s ubmitting a MAP2 mob ile applic ation for VA V&V tes ting and c ompliance reviews, t he MAP2 HP ES SQA tea m will wor k with the MAP2 HPES Release M anager to ensure the appropria te release forms are filled ou t accurate ly and ver ify suppor ting docum entation i s included .
	186	The follow ing is a l ist of sam ple releas e forms or supportin g document s that wil l be revie wed. This is not an all-inclus ive list. The applic ation-leve l MTP will contain t he list of release f orms and s upporting documents that will be reviewe d:
	187	V&V Intake Request F orm
	188	V&V Applic ation Prof ile Form
	189	Concept Pa per
	190	Code Revie w Question naire
	191	Enterprise Security Questionna ire
	192	Business R equirement s Document (BRD)
	193	Privacy an d Security Checklist
	194	Section 50 8 Review F orm
	195	Release Ma nagement P lan
	196	Requiremen ts Traceab ility Matr ix
	197	RSD MA Add endum (Use r Stories/ Acceptance Criteria)
	198	Software Q uality Ass urance (SQ A) Checkli st - only required i f VistA ch anges are made
	199	SQA Test S cripts/Res ults and D efect Log
	200	System Des ign Docume nt MA Adde ndum
	201	System Sec urity Plan Addendum
	202	User Guide
	203	Again, thi s document ation is n eeded prio r to entra nce into t he Complia nce Review phase. Th e Complian ce Review phase begi ns when th e applicat ion code i s handed o ff to V&V. For more informatio n about VA Complianc e Review, please ref er to the VA Mobile Applicatio n Developm ent web si te (https: // DNS /content/v a-app-comp liance-rev iew). The HPES SQA T est team w ill assist the MAP2 Release Ma nager with review of these doc uments pri or to thei r turnover to the V& V complian ce review body.
	204	Middleware Updates a nd Additio ns
	205	Some mobil e applicat ions may h ave unique requireme nts based on their a rchitectur e defined in the SDD Addendum and SSP Ad dendum. Fo r example, if the de ployment o f a mobile applicati on require s a new ap plication server, su ch as Node JS or adde d capacity to the We bLogic clu ster, then testing m ust occur on these n ew infrast ructure pi eces. Spec ifically, security s cans must be execute d to ensur e that the new serve rs are bui lt to the expected s ecurity co nfiguratio ns. The MA E Maintena nce Team m ust execut e these te sts at the direction of their Contractin g Officer' s Represen tative (CO R).
	206	Overview o f Test Exc lusions
	207	Note: Addi tional inf ormation f or the Ove rview of T est Exclus ions secti on (Sectio n 2.2 in t his docume nt) will b e added/up dated in f uture iter ations, as needed.Th e followin g componen ts and fea tures (and combinati ons of com ponents an d features ) will not be tested :
	208	Testing of enhanceme nts to VA enterprise services – mobile a pplication s use VA e nterprise services, such as th e Data Acc ess Servic es (DAS), VistA, and the Clini cal Data W arehouse ( CDW). Whil e testing of the mob ile applic ation invo lves testi ng the int egration w ith these enterprise services, it does n ot include testing o f the serv ice itself . This is the respon sibility o f the deve lopment/SQ A team for the enter prise syst em or serv ice.
	209	Non-functi onal requi rements th at are not applicabl e to the n ew or enha nced appli cation dev eloped by the MAP2 D evelopment team.
	210	Non-functi onal requi rements th at are not testable.
	211	Test Appro ach
	212	Note: Addi tional inf ormation f or the Tes t Approach section o f this doc ument will be added/ updated in future it erations a s needed.
	213	The MAP2 p rogram tea m will be utilizing an Agile D evelopment Methodolo gy (ADM) a s well as using best practices in agile testing te chniques f or all app lications developed as part of the MAP2 program.
	214	Developers assigned to a Scrum team will perform p roduct com ponent or unit testi ng on thei r local de velopment workstatio n and comp onent-inte gration te sting for each sprin t iteratio n in the M AE Develop ment envir onment. Ne xt, the HP ES SQA tes ter assign ed to the Scrum team will perf orm in-sco pe feature (function al) verifi cation and non-funct ional test ing in the MAE Devel opment-Dem o environm ent, which involves testing ag ainst the user story 's accepta nce criter ia or the non-functi onal requi rements (N FRs) accep tance crit eria. In a ddition, a parallel process of regressio n testing will occur throughou t the spri nt iterati on. This i nvolves re -running t he automat ed and/or manual pro duct-compo nent tests and featu re verific ation test s from the current s print iter ation and previous i terations via a cont inuous int egration f ramework. This level of testin g will be completed in the MAE Developme nt and Dev elopment-D emo enviro nments.
	215	Members of the MAP2 HPES SQA T est team w ill perfor m the Syst em Test le vel, which starts on ce the fir st user st ory is rea dy for thi s test lev el. System testing w ill consis t of execu ting funct ional, end -to-end, s ystem-inte gration, f ull regres sion, perf ormance, e tc. in the MAE Devel opment-Tes t environm ent.
	216	Testing Pr ocesses
	217	The follow ing testin g processe s will be employed b y the MAP2 HPES SQA Test team:
	218	Develop th e Master T est Plan
	219	Establish the Test E nvironment s
	220	MAE Develo pment
	221	MAE Develo pment-Demo
	222	MAE Develo pment-Test
	223	Prepare Te st Scenari os, Cases, Scripts a nd Data
	224	Conduct th e Tests
	225	Verify Tes t Results
	226	Document T est Result s
	227	Update the Test Stat us
	228	Identify a nd Resolve Discrepan cies
	229	Retest rem ediated de fects
	230	Representa tives of t he user-ba se perform user acce ptance tes ting, and independen t VA V&V T est or Com pliance Re view teams perform v erificatio n and vali dation tes ting and c ompliance body revie ws. User a cceptance and compli ance revie w testing will take place in M AE environ ments, suc h as the I ntegration or Pre-Pr oduction e nvironment s. These t est enviro nments wil l be setup and maint ained by V A resource s. While t he strateg y is stand ard to bes t practice s in testi ng, there are some u nique aspe cts to the mobile an d MAP2 tes t approach .
	231	The MAP2 p rogram wil l be follo wing an Ag ile Develo pment Meth odology (A DM), which integrate s testing into the e ntire deve lopment pr ocess vers us a separ ate test p hase at th e end of t he develop ment cycle as well a s promotes constant communicat ion with t he busines s owner's team. This yields ap plication software t hat implem ents the i ntended re quirements and will not yield many requi rement sur prises in the downst ream testi ng levels.
	232	The ADM fo llowed by the MAP2 p rogram tea m promotes the follo wing Agile test-rela ted strate gies:
	233	HPES SQA t ester invo lvement in user stor y refineme nt and spe cification of accept ance crite ria as par t of story definitio n
	234	Automate p roduct com ponent, co mponent-in tegration, system an d acceptan ce testing as much a s possible
	235	Incorporat ion of aut omated tes ts into th e continuo us build e nvironment
	236	Small (min imal) hand -offs betw een Scrum team (deve lopers, bu siness ana lyst, test ers) membe rs (they w ork togeth er)
	237	Lightweigh t (fast-tu rnaround) defect tra cking and management (defects are identi fied and f ixed withi n the spri nt)
	238	Explorator y testing
	239	Test in th e same spr int in whi ch the fea ture is de veloped
	240	Track test coverage
	241	Incorporat ion of tes ting into the "Defin ition of D one"
	242	Considerat ion of how various f acets of t esting wil l be perfo rmed durin g the Rele ase Cycle (e.g., str onger emph asis on pe rformance testing du ring 'hard ening spri nt iterati ons').
	243	Separate b lack-box ( behavioral ) unit tes ting from white-box (code-stru cture base d) unit te sting:
	244	Product co mponent or unit test ing is typ ically the responsib ility of d evelopers. However, for agile projects, it is reco mmended th at the res ponsibilit y for whit e-box test ing be ass igned to d evelopers, while the testers c an focus o n black-bo x (feature -driven) u nit testin g concurre ntly. This provides complement ary but co mprehensiv e focus to unit test ing while also enabl ing early detection of defects that woul d typicall y be found in later test level s (e.g., s ystem test ing) and r educes cyc le time fo r defect f ixes and r ework.
	245	Integrate Automated Tests with the Conti nuous Inte gration Bu ild Enviro nment
	246	Automated tests shou ld run aga inst the c ontinuous build envi ronment. T ypically, this is se t-up eithe r on a uni que integr ation serv er or done in-memory so that t hese tests do not im pact other manual te sters.
	247	Guidelines for selec tion of re gression t est cases for automa tion are p rovided be low. These should be considere d when usi ng the ass essed impl ementation risk for each featu re to dete rmine the candidate test cases for autom ation:
	248	Feature Sm oke tests to verify that the d efinition of 'done' is achieve d, testing the basic transacti ons. Give these auto mated test suites to the devel opers as t heir 'exit ' criteria . Testers run these again in t he integra tion testi ng environ ment
	249	Integratio n Smoke te sts (build smoke tes ts) to ver ify that t he code is ready for system te sting (har dening spr int)
	250	Solution S moke tests to verify that the key featur es develop ed in one sprint are not broke n in subse quent spri nt cycles
	251	The compli ance revie ws that mu st be perf ormed on a mobile ap plication are define d based on the level of risk t o VA. The risk level is define d as one o f the four levels, a s shown in Table 2. The compli ance bodie s that mus t review t hose appli cations ar e also sho wn.
	252	Table 2: R isk Level Definition
	253	Mobile App lication C lassificat ion
	254	1a - Very Low: Non- OI&T**
	255	1 - Very L ow
	256	2 - Low
	257	3 - Medium
	258	4 - High
	259	Compliance Review Bo dy
	260	Does not u tilize VA resource o r OI&T Fun ding
	261	Does not u tilize VA resources
	262	Read only access to VA resourc es
	263	Write acce ss to VA o r external resources
	264	Read and/o r write ac cess to VA or extern al sensiti ve resourc es
	265	V&V
	266	Required
	267	Required
	268	Required
	269	Required
	270	Required
	271	Business O wner Accep tance
	272	Required
	273	Required
	274	Required
	275	Required
	276	Required
	277	Patient Sa fety Asses sment (OIA )
	278	Required
	279	Required
	280	Required
	281	Required
	282	Required
	283	508 Access ibility (O I&T)*
	284	Required
	285	Required
	286	Required
	287	Required
	288	Required
	289	Code Revie w
	290	Required
	291	Required
	292	Required
	293	Required
	294	Required
	295	Usability Testing (O IA)
	296	Required
	297	Required
	298	Required
	299	Required
	300	Required
	301	Mobile App lication C lassificat ion
	302	1a - Very Low: Non- OI&T**
	303	1 - Very L ow
	304	2 - Low
	305	3 - Medium
	306	4 - High
	307	Compliance Review Bo dy
	308	Does not u tilize VA resource o r OI&T Fun ding
	309	Does not u tilize VA resources
	310	Read only access to VA resourc es
	311	Write acce ss to VA o r external resources
	312	Read and/o r write ac cess to VA or extern al sensiti ve resourc es
	313	User Inter face (OIA)
	314	Required
	315	Required
	316	Required
	317	Required
	318	Required
	319	VA Brandin g (OPIA)
	320	Required
	321	Required
	322	Required
	323	Required
	324	Required
	325	Privacy an d Applicat ion Securi ty (OIA)
	326	Required
	327	Required
	328	Required
	329	Required
	330	Required
	331	Sustainmen t Plan*
	332	Required
	333	Required
	334	Required
	335	Required
	336
	337	System Per formance I mpact Asse ssment (OI &T)*
	338	Required
	339	Required
	340	Required
	341
	342
	343	Enterprise Security
	344	Required
	345	Required
	346	Required
	347
	348
	349	Data and T erminology Standards Complianc e (OIA)*
	350	Required
	351	Required
	352
	353
	354
	355	Not Requir ed for Fie ld Test
	356
	357
	358
	359
	360
	361
	362	In order t o qualify for this c ategory, t he Mobile Applicatio n (MA) mus t meet the following criteria:
	363	Does not a ccess Pers onally Ide ntifiable Informatio n (PII) or Personal Health Ide ntifiers ( PHI) data
	364	Does not a ccess the VA network
	365	Does not r ead and/or write to VA or exte rnal datab ase
	366	Does not s tore data in any VA database
	367	Does not u tilize the VA Mobile Framework
	368	Does not r eceive fun ding (incl uding sust ainment an d future r elease fun ding) from the Offic e of Infor mation and Technolog y (OI&T)
	369	Note: The VAMF is co nsidered a VA resour ce. With t his defini tion, the following is implied :
	370	If an appl ication is hosted in the VA Mo bile Frame work (VAMF ), it cann ot be a Ve ry Low app . It is a t least a Low if it only reads pages wit hin the VA MF or read s data wit hin the VA MF.
	371	If an appl ication us es a servi ce in the VAMF, but is not ins talled in the VAMF, then it mu st be at l east a Low app.# The MAE provi des develo pment, tes t, integra tion, and other envi ronments t hat provid e standard ized mobil e technolo gies and t oolsets. T he Web and Mobile So lutions (W MS) team m aintains t he MAE. Th e WMS team has defin ed and con tinues to define met hods for u sing the t ools to st reamline t he process es and mak e the comp liance ste ps and doc umentation easier to find and read. Thes e processe s are inte nded to sp eed up the testing p rocess but still mai ntain qual ity in ord er to ensu re the pro ducts work as intend ed and pas s governme nt regulat ions, such as Nation al Institu te of Stan dards and Technology (NIST) se curity and privacy g uidelines, 508 compl iance, and VA patien t safety r egulations . Tools, s uch as JIR A and Conf luence (wi ki), provi de the tra cking mech anisms and document management that allo w for thes e streamli ned proces ses.
	372	Each mobil e applicat ion will d ocument te st scripts and test cases in a Microsoft Excel wor kbook and include th e workbook as an att achment to the appli cation's M aster Test Plan (MTP ). The Req uirements Traceabili ty Matrix (RTM) for the mobile applicati on will be updated t o map the applicable test case identific ation (ID) for the t est cases that provi des test c overage fo r a user s tory. The MTP will d ocument th e overarch ing approa ch and tes t environm ent such t hat the te st cases a nd scripts contained as an att achment to the MTP n eed to onl y address specifics of the mob ile applic ation bein g tested. The HPES S QA Test Te am will co mplete the test case s and test scripts f or all app licable fu nctional a nd non-fun ctional re quirements in scope for the mo bile appli cation bei ng tested.
	373	Currently, the MAP2 program te am is fore casting th at it shou ld take fi ve sprint iterations to comple te all dev elopment a nd testing activitie s (up thro ugh the Sy stem Test level) for each new or enhance d applicat ion. With this in mi nd, system testing w ill be a p arallel te sting acti vity that will begin once the user stori es from Sp rint 1 for a given a pplication have comp leted feat ure verifi cation tes ting and m et the def inition of "done" by the VA Bu siness Own er of that applicati on. At the earliest, system te sting for a given ap plication will begin no earlie r than Spr int 2 and will be co mpleted by Sprint 5.
	374	Product Co mponent Te st and Com ponent-Int egration T est
	375	The MAP2 d evelopers will perfo rm product component testing o r unit tes ting and c omponent-i ntegration testing u sing their local dev elopment c omputer wo rkstation and the MA E Developm ent enviro nment. In traditiona l software developme nt, "units " were mor e distinct elements that were testable u sing unit test drive rs. The un it test dr ivers woul d test ind ividual pi eces of co de and rep ort the re sults and separate t esting of combined u nits const ituted com ponent-int egration t esting.
	376	In the mob ile techno logy space , the line between u nits is bl urred, but unit test ing by the developer is still required. MAP2 has c ombined th ese two te st levels into a sin gle test l evel. Deve lopers are expected to test th e componen ts of the applicatio n that the y are resp onsible fo r, and dev elopers as signed to the Scrum team need to test th e applicat ion compon ents as a whole befo re turning the appli cation ove r to the S crum HPES SQA Test t eam member for each sprint.
	377	The Produc t Componen t Test and Component Integrati on Test wi ll be know n as devel oper led t esting and will occu r on the M AP2 Develo per's comp uter works tation and the MAE D evelopment environme nt. Both e nvironment s contain mock servi ces for sy stem inter faces.
	378	After prod uct compon ent and co mponent-in tegration testing is completed , the appl ication co de will be promoted to the Dev elopment-D emo enviro nment. The MAP2 HPES SQA Test team membe r assigned to the sc rum team w ill perfor m feature verificati on testing and work with the o ther scrum team memb ers to ens ure all de fects iden tified are fixed and retested or resolve d.It is ex pected tha t manual t esting wil l occur du ring this level, but that auto mated test scripts w ill be wri tten by th e MAP2 HPE S Developm ent team t o facilita te this te st level. The MAP2 H PES Develo pment and Test Manag ers will h elp determ ine the sc ope of aut omated tes t script d evelopment for this test level .
	379	System Tes ts
	380	MAP2 HPES SQA Test t eam member s will tes t all in-s cope funct ional and non-functi onal requi rements in the MAE D evelopment -Test envi ronment.
	381	The MAE De velopment- Test envir onment wil l contain live inter faces with as many V A enterpri se systems and/or se rvices as possible t o perform valid syst em tests. Mock servi ces will o nly be use d where ab solutely r equired (f or example , if a giv en VA syst em does no t have a t est enviro nment that can be co nnected to the MAE). The HPES SQA Test t eam will b e in contr ol of the test data, and devel opers shou ld not hav e access t o this env ironment t o make cha nges. It m ay be requ ired for d evelopers to have re ad access such that they can d ebug issue s when the y arise in system te sts.Specif ically, th e HPES SQA Test team will perf orm the fo llowing fu nctions:
	382	1. Write the test s cripts inc luding:
	383	Test Case Descriptio ns
	384	Pre-condit ions/Post Conditions
	385	Test Cases
	386	Test Scrip t specific ations inc luding:
	387	Procedure steps
	388	Verificati on Points (VP)
	389	Expected R esults (ER )
	390	Comments
	391	2. Update the Requir ements Tra ceability Matrix cre ated earli er by an B usiness or Technical Analyst a dding the Test Case ID to the matrix
	392	3. Execute the test scripts
	393	4. Report results ba sed on the actual re sults comp ared to th e expected results
	394	5. Create bug issues in JIRA t o document defects
	395	6. Perform testing o f remediat ed defects :
	396	As fixes t o the defe cts are pl anned (and put into the backlo g in JIRA) , the Test Team will track the m.
	397	When fixed , the defe cts will b e retested by the Te st Team, c losed, or reopened d epending o n the outc ome of tes t executio n.
	398	Regression testing w ill occur on new bui lds to ens ure that f unctionali ty that wa s working is still w orking.
	399	Update the test scri pts with p ossible ne w tests or new steps to the ex isting tes t scripts as the sof tware is m odified.
	400	7. Create artifacts needed for release o f the soft ware, such as:
	401	Defect Log (the defe ct log can be a repo rt output from JIRA)
	402	Test Execu tion Summa ry that pr ovides ove rall stati stics and results ab out the Sy stem Test level (thi s report w ill be rep orted from the Test Case/Scrip t workbook )
	403	In Agile p rocesses, the SQA Te st team is involved throughout the proce ss includi ng testing of delive red sprint s. However , there is still a s pecific Sy stem Test sprint or sprints in cluded in the agile process to allow for verificat ion of the functiona l and non- functional requireme nts prior to turnove r to VA Co mpliance R eviews and V&V Test processes.
	404	Performanc e and Load Test
	405	Note: Addi tional inf ormation f or the Per formance a nd Load Te st section will be a dded/updat ed in futu re iterati ons as nee ded.
	406	Performanc e and load testing w ill be per formed by members of the MAP2 HPES Devel opment and SQA Test teams as r equired by the MAP2 Performanc e Work Sta tement (PW S) documen t (dated 8 /29/2014). The MAP2 program te am will re view all a pplicable VA Enterpr ise NFRs r elated to performanc e and load test requ irements. If it is d etermined that a per formance a nd/or load testing i s required , performa nce and/or load test cases and scripts w ill be dev eloped. Th e determin ation to e xecute a P erformance and/or Lo ad Test wi ll be base d on facto rs, such a s the numb er of user s to be us ing the ne w applicat ion, the t ypes of se rvices tha t will be executed w ithin the environmen t, and the status of prior per formance t ests (i.e. , have the functions been test ed before? ). If it i s determin ed that a Performanc e and Load Test is r equired, t he followi ng are hig h-level go als to be met for th e Performa nce and Lo ad Test:
	407	Validate t hat docume nted Criti cal Succes s Criteria around pe rformance of the int egrated sy stem and i nfrastruct ure, which map to Cr itical Suc cess Facto rs that ar e visible to and agr eed with t he custome r, have be en satisfi ed
	408	Assure the system wi ll perform as requir ed under e xpected us age levels
	409	Identify p oints of s ystem degr adation or bottlenec ks
	410	Identify s ystem capa city or li mitations on specifi c componen ts
	411	Reduce imp lementatio n risk
	412	Identify c auses of p oor perfor mance to t he busines s function s
	413	A System P erformance Impact As sessment w ill be com pleted bas ed on the performanc e test res ults or th e fact tha t no perfo rmance imp act is exp ected base d on the S DD Addendu m review.
	414	User Funct ionality T est
	415	Note: Addi tional inf ormation f or the Use r Function ality Test section w ill be add ed/updated in future iteration s as neede d.
	416	User Funct ionality T est (UFT – also know n as User Acceptance Test) req uires stak eholders t o test the mobile ap plication prior to r elease. Th ese tests are typica lly perfor med by mem bers of th e business sponsor o rganizatio n that hav e the know ledge of t he functio nal requir ements, bu t they can be perfor med by act ual end us ers. The o bjective i s to confi rm that th e product meets the business n eed or bus iness acce ptance cri teria. It is the res ponsibilit y of the b usiness sp onsor to d efine the contents o f the UFT and the me thods for performing it. UFT t esting wil l be perfo rmed in th e MAE Inte gration an d Pre-Prod uction env ironments. The MAP2 Developmen t and SQA Test team will suppo rt this te st level a s required .
	417	Since the mobile app lications are develo ped using an Agile p rocess, bu siness own er represe ntatives s hould be i nvolved in the devel opment pro cess, and it is more likely th at UFT wil l be less intense an d short in duration.
	418	It is expe cted that user funct ionality t esting wil l be done using the user stori es develop ed and ref ined early in the de velopment phase of t he project .
	419	Enterprise System En gineering Testing
	420	V&V Test
	421	V&V Test w ill be use d to valid ate the fe atures or functional and non-f unctional testing co mpleted by MAP2 HPES SQA Test team. Memb ers of the HPES SQA Test team will work with the M AP2 Releas e Manager to ensure the applic ation code and any r equired ar tifacts (i .e., Test Cases, Tes t Scripts, updated R TM, etc.) are ready for releas e to the V &V team.
	422	ESE Releas e Process
	423	Members of the MAP2 HPES SQA T est team w ill coordi nate with the MAP2 R elease Man ager to en sure requi red docume ntation is submitted to the En terprise S ystems Eng ineering ( ESE) Compl iance body responsib le for com pleting th is type of testing o n the mobi le applica tion-under -test.
	424	Initial Op erating Ca pability E valuation
	425	The Initia l Operatio n Capabili ty (IOC) E valuation will be ac complished through l imited rel ease of ap plications to real u sers in th e producti on environ ment. The mobile app lication m ust pass a ll require d complian ce body re views and the V&V Te st in orde r to be re leased to the real e nd users. The defini tion of th e required complianc e bodies i s dependen t on the m obile appl ication. F or example , mobile a pplication s that are "Very Low " systems do not hav e to be as sessed by the Privac y complian ce team. T he MAP2 Re lease Mana gement pro cesses wil l be follo wed before releasing to produc tion for I OC Evaluat ion.
	426	Testing Te chniques
	427	Risk-based Testing
	428	Risk-based testing i s a techni que for pr ioritizing testing b ased on te sting the highest ri sk items f irst and c ontinuing to test do wn the ris k prioriti zation lad der as the testing s chedule pe rmits. In an Agile p rocess, it is typica l to imple ment high- risk items in early sprints. T he MAP2 Sc rum teams (i.e., Dev elopers) w ill perfor m their in ternal pro duct compo nent and c omponent-i ntegration testing o n these ea rly sprint s and cont inue testi ng as the agile cycl es continu e. The MAP 2 Scrum te ams will i dentify hi gh-risk ar eas and th e MAP2 Scr um teams ( i.e., HPES SQA Teste rs) will b e involved early in the proces s to test these item s prior to the final developme nt sprint.
	429	For System Testing, V&V Testin g, and Com pliance Te sting, all functiona l, non-fun ctional, a nd complia nce requir ements mus t be met. While risk based tes ting can b e performe d to ident ify high-r isk issues first, th ere is not as much v alue in th is approac h as with the User S tory featu res testin g done by the MAP2 S crum teams . In the e nd, all re quirements must be t ested in s ome manner even if i t is throu gh inspect ion versus actual te sting.
	430	Enterprise Testing
	431	Security T esting
	432	The MAP2 H PES Develo pment and MAP2 HPES SQA Test t eam will d evelop tes ts to vali date the s ecurity re quirements and to en sure readi ness for t he indepen dent testi ng perform ed by the External A ssessment Services ( EAS) team. The EAS t eam perfor ms Web App lication S ecurity As sessments (WASAs) th at are an in-depth p enetration test for common vul nerabiliti es, such a s SQL Inje ction, Aut horization Bypass, a nd Cross-S ite-Script ing (CSS). WASAs can not be sta rted until the EAS t eam receiv es a compl eted quest ionnaire w ith workin g test acc ounts and a full dir ectory lis ting. The HPES SQA T est team w ill ensure applicabl e applicat ion Unifor m Resource Locators (URLs) and test acco unts work before the questionn aire is su bmitted to the EAS t eam. In ad dition to the EAS ev aluation, the Networ k Security Operation s Center ( NSOC) will scan the applicatio n and the code for v ulnerabili ties. The MAP2 HPES Developmen t team nee ds to comp lete the N SOC Intake Questionn aire prior to submit ting the a pplication for the E AS evaluat ion. This penetratio n test is particular ly importa nt for tho se mobile applicatio ns that re ad and wri te data to VA system s. All MAP 2 applicat ions are c lassified as "High;" so, these scans wil l be requi red.
	433	Privacy Te sting
	434	VHA Privac y and Appl ication Se curity off ices ensur e that mob ile applic ations adh ere to Pri vacy regul ations and statutes as well as VA policy . The MAP2 HPES Deve lopment an d MAP2 HPE S SQA Test team use the Privac y and Secu rity check list to de termine if data is s tored, tra nsmitted, or entered by the us er, provid er or empl oyee. The MAP2 HPES Developmen t and MAP2 HPES SQA Test team also deter mine if th ere is sen sitive inf ormation, such as PH I or PII s tored on t he applica tion. If i t is, the Data Secur ity branch determine s how the applicatio n-under-te st protect s the info rmation.
	435	Section 50 8 Complian ce Testing
	436	The Office of Inform ation and Technology (OIT) Sec tion 508 t ests mobil e content to verify that it co mplies wit h Section 508 standa rds includ ing 1194.2 1 (Softwar e and Plat forms), 11 94.22 Web, 1194.24 ( Multimedia ), and 119 4.31 Funct ional Requ irements. They work to facilit ate access to mobile technolog ies for an yone with disabiliti es.
	437	The Sectio n 508 Offi ce will ma nually tes t VA platf orm-specif ic applica tions dire ctly on th e device u sing its b uilt-in ac cessibilit y features and suppo rted metho ds. For ex ample, the y test wit h VoiceOve r on iOS a nd Talkbac k on Andro id.
	438	The Sectio n 508 Offi ce will pe rform some automated testing o n web-base d content, but they also perfo rm manual tests to e nsure the (mobile ap plication) properly implements accessibi lity techn iques on t he devices browsers.
	439	The MAP2 H PES Develo pment and MAP2 HPES SQA Test t eams will perform 50 8 Complian ce testing prior to initiating the Secti on 508 Pro gram Offic e to perfo rm their i ndependent testing.T he MAP2 HP ES SQA tea m will wor k with the MAP2 Rele ase Manage r to regis ter MAP2 a pplication s as early in the Pl anning or Developmen t phases a s possible .
	440	Multi-Divi sional Tes ting
	441	Multi-Divi sional tes ting is re quired to ensure tha t all appl ications w ill operat e in a mul ti-divisio n or multi -site envi ronment re cognizing that an en terprise p erspective while ful l supporti ng local h ealth care delivery. If multi- divisional requireme nts are pr esent in a n applicat ion's BRD, the MAP2 HPES Devel opment and MAP2 HPES SQA Test teams veri fy and val idate that the appli cation und er test co mplies wit h the mult i-division al require ments stat ed prior t o initiati ng the app licable Co mpliance b ody to per form their independe nt testing .
	442	Test Types
	443	Table 3 li sts the fo llowing te st types t hat can be performed under MAP 2. While a ll of thes e test typ es do not list the M AP2 HPES D evelopment or MAP2 H PES SQA Te st teams, the MAP2 H PES Develo pment Team and the c orrespondi ng MAP2 HP ES SQA Tes t team mus t ensure t hat the mo bile appli cation is compliant in the par ticular ar ea before moving the mobile ap plication past the S ystem Test level.Tab le 3: Test Types
	444	Test Types
	445	Party Resp onsible
	446	Access Con trol Testi ng
	447	MAP2 HPES Developmen t and MAP2 HPES SQA Teams, VA V&V Team
	448	Build Veri fication T esting
	449	MAP2 HPES Developmen t Team
	450	Code Cover age Testin g
	451	MAP2 HPES Developmen t Team, VA Complianc e Body
	452	Compliance Testing
	453	VA V&V Tea m and Comp liance Bod ies
	454	Component Integratio n Testing
	455	MAP2 HPES Developmen t Team
	456	Data and D atabase In tegrity Te sting
	457	MAP2 HP De velopment and SQA Te ams, VA V& V Team
	458	Documentat ion Testin g
	459	VA V&V Tes t Team
	460	Error Anal ysis Testi ng
	461	MAP2 HPES Developmen t and MAP2 HPES SQA Teams, VA V&V Team
	462	End-to-End Testing
	463	MAP2 HPES Developmen t and MAP2 HPES SQA Teams, VA V&V Team
	464	Functional Testing
	465	MAP2 HPES Developmen t and MAP2 HPES SQA Teams, VA V&V Team
	466	Informatio n Assuranc e Testing
	467	MAP2 HPES Developmen t and MAP2 HPES SQA Teams, VA V&V Team
	468	Installati on Testing
	469	VA MAE Mai ntenance T eam
	470	Integratio n Testing
	471	MAP2 HPES Developmen t and MAP2 HPES SQA Teams, VA V&V Team
	472	Integrated Performan ce and Loa d Testing
	473	MAP2 HPES Developmen t and MAP2 HPES SQA Teams
	474	Privacy Te sting
	475	MAP2 HPES Developmen t and MAP2 HPES SQA Teams, VA V&V Team a nd VA Comp liance Bod ies
	476	Product Co mponent Te sting
	477	MAP2 HPES Developmen t Team
	478	Regression Testing
	479	MAP2 HPES Developmen t and MAP2 HPES SQA Teams, VA V&V Team
	480	Section 50 8 Complian ce Testing
	481	MAP2 HPES Developmen t and MAP2 HPES SQA Teams, VA V&V Team, and VA Com pliance Bo dies
	482	Security T esting
	483	MAP2 HPES Developmen t and MAP2 HPES SQA Teams, VA V&V Team, and VA Com pliance Bo dies
	484	Smoke Test ing
	485	MAP2 HPES SQA Test T eam, VAV&V Team
	486	Usability Testing
	487	VA V&V Tea m and Comp liance Bod ies
	488	User Funct ionality T esting
	489	VA Stakeho lders
	490	User Inter face Testi ng
	491	MAP2 HPES Developmen t and MAP2 HPES SQA Teams, VA V&V Team
	492	Productivi ty and Sup port Tools
	493	Table 4 de scribes th e tools th at will be employed to support this Mast er Test Pl an.Table 4 : Tool Cat egory or T ypes
	494	Tool Categ ory or Typ e
	495	Tool Brand Name
	496	Vendor or In-house
	497	Version
	498	Test Manag ement
	499	MS Excel
	500	Microsoft
	501	2013
	502	Defect Tra cking
	503	JIRA
	504	Atlassian
	505	6.2
	506	Code Analy zer
	507	Fortify
	508	Atlassian
	509	6.2
	510	Document M anagement
	511	Confluence (wiki)
	512	Atlassian
	513	6.2
	514	Performanc e and Load Testing
	515	Apache JMe ter
	516	Open Sourc e
	517	TBD
	518	Configurat ion Manage ment
	519	Stash/Git
	520	Atlassian
	521	6.2
	522	DBMS tools
	523	Oracle & M ongoDB
	524	Oracle and 10gen
	525	TBD
	526	Functional Test Auto mation
	527	Selenium
	528	Vendor
	529	2.5.0
	530	Web Servic e Test Aut omation
	531	SoapUI
	532	Open Sourc e
	533	TBD
	534	Database Q uery Tools
	535	TOAD, SQLP lus
	536	Quest, Ora cle
	537	TBD
	538	Test Crite ria
	539	Process Re views
	540	The Master Test Plan undergoes three rev iews:
	541	Peer Revie w – upon c ompletion of the Mas ter Test P lan
	542	MAP2 HPES Technical Writer Rev iew
	543	Formal Rev iew – afte r the HPES MAP2 Prog ram and Pr oject Mana ger approv es the Mas ter Test P lan
	544	The Master Test Plan does serv e as an in put artifa ct used fo r Mileston e 1 and Mi lestone 2 Reviews of the Devel opment pha se.For mor e informat ion on the milestone reviews a ssociated with testi ng, see th e required artifacts and templ ates for P MAS as par t of VA's Mobile App lication D evelopment Life Cycl e.
	545	Pass/Fail Criteria
	546	Per the Ag ile Develo pment Meth odology (A DM) follow ed by the MAP2 Scrum team, all defect(s) found for a given u ser story will be fi xed and re tested dur ing the sp rint the f eature(s) were devel oped. If f or some re ason the d efect(s) f or a certa in user st ory cannot be fixed and retest ed prior t o sprint c ompletion, the defec t and appl icable use r story wi ll be addr essed in a subsequen t sprint.
	547	For the Sy stem Test level, pas s criteria is define d in each specific t est case, but in gen eral, a te st is cons idered as passed whe n the func tionality meets the requiremen ts as defi ned with n o Level 1, Level 2, or Level 3 defects i dentified. If any te st case wi thin the o verall tes t fails, t hen the wh ole test f ails. It i s typical that a pro duct will still be r eleased wi th some Le vel 3 and Level 4 de fects, but it must b e approved by the bu siness own er and the release p rocesses i n order to be approv ed out to production . The open issues ar e provided as part o f the rele ase docume ntation. A test is c onsidered failed whe n the func tionality does not m eet the re quirements as define d.
	548	Suspension and Resum ption Crit eria
	549	The Projec t Manager at the rec ommendatio n of the M AP2 HPES S QA Test te am has the authority to suspen d testing. Testing w ill be sus pended if:
	550	Test accou nts or ver ifiable it ems are un available or become unstable
	551	A failure occurs tha t prevents the test from conti nuing or i nvalidates any addit ional test ing to be performed or corrupt s the data base
	552	A defect i s discover ed that co rrupts the data with in the dat abase in s uch a way that proce eding woul d cause se vere damag e to the t est enviro nment
	553	The compon ent being tested fai ls or a ma jor compon ent fails. A major c omponent f ailure is one that o ne can rea sonable as sume will result in other test case fail ures. If t here are s o many def ects that continuing testing i s a waste of resourc es, testin g will be suspended.
	554	Testing wi ll resume when the c omponent o r code has been repa ired, rebu ilt and ve rsioned, u nit tested , gone thr ough syste m test, an d is insta lled into the test e nvironment by the MA E maintena nce team. If the cau se of the suspension is due to either un stable or unavailabl e test env ironment, testing wi ll resume when the t est enviro nment beco mes stable and/or av ailable.
	555	Acceptance Criteria
	556	The follow ing activi ties must be perform ed to comp lete the t est and va lidation p rocess:
	557	Updates to test docu mentation have been completed and are un der config uration ma nagement c ontrol on the mobile applicati on wiki pa ge
	558	All requir ed test ty pes have b een comple ted (i.e., Functiona l, 508, Se curity and Regressio n, etc.)
	559	Test cases have been executed according to the tes t plan and any devia tions are documented and appro ved
	560	All compli ance bodie s have app roved the release or stated th at their p articipati on is not required f or the spe cified mob ile applic ation bein g tested
	561	All defect s have bee n analyzed and chang ed to the appropriat e state
	562	All closed defects h ave an ass igned defe ct root ca use
	563	After comp letion of the above activities , the acce ptance cri terion for release o f the appl ication is :
	564	For System Test, the applicati on has no Level 1, L evel 2, or Level 3 d efects exc ept as spe cifically accepted b y the busi ness owner .
	565	All defect s that are in the pr oduct are documented in the so ftware ano malies rep ort as kno wn defects and accep ted by the business owner.
	566	All MAP2 R elease Man agement ac tivities a re carried out.
	567	The busine ss sponsor and requi red compli ance bodie s have sig ned off on the relea se.
	568	Test Deliv erables
	569	Table 5 li sts the te st deliver ables that will be i ncluded fo r each app lication d eveloped u nder the M AP2 progra m. Table 5 : Test Del iverables
	570	Test Deliv erables
	571	Program/ A pp level?
	572	Responsibl e Party
	573	Master Tes t Plan
	574	Program
	575	MAP2 HPES SQA Test M anager
	576	Master Tes t Plans (I terative)
	577	App level
	578	MAP2 HPES SQA Test P lanner/Eng ineer
	579	Test Execu tion Risks
	580	App level
	581	MAP2 HPES SQA Test M anager and Test Plan ner/Engine er
	582	Test Sched ule
	583	App level
	584	MAP2 HPES SQA Test M anager and Project M anager
	585	Test Cases /Test Scri pts
	586	App level
	587	MAP2 HPES Developers and SQA T esters
	588	Test Data
	589	App level
	590	MAP2 HPES Developers and SQA T esters
	591	Test Envir onment(s)
	592	Program
	593	MAP2 HPES SQA Test M anager
	594	Test Evalu ation Summ ary
	595	App level
	596	MAP2 HPES SQA Test P lanner/Eng ineer for Sprint Ite rations an d System T est HPES V &V Team fo r V&V Test s and Comp liance Rev iews
	597	Traceabili ty Report or Matrix
	598	App level
	599	MAP2 HPES SQA Test P lanner/Eng ineer for Sprint Ite rations an d System T est VA V&V Team for V&V Tests and Compli ance Revie ws
	600	Test Sched ule
	601	Table 6 li sts the ma jor testin g mileston es for the MAP2 prog ram. Refer to each m obile appl ications m aster proj ect schedu le for cur rent plann ed start a nd complet ion dates for testin g mileston es.Table 6 : Testing Milestones
	602	Testing Mi lestones
	603	Responsibl e Party
	604	Deliver In itial MTP and MTP Ch ecklist fo r all MAP2 Mobile Ap plications
	605	MAP2 HPES Test Manag er
	606	Complete S print 1 Us er Story/U nit Test/S QA Testing
	607	MAP2 HPES Developers and MAP2 HPES SQA T esters
	608	Complete S print 2 Us er Story/U nit Test/S QA Sprint Testing
	609	MAP2 HPES Developers and MAP2 HPES SQA T esters
	610	Complete S print 3 Us er Story/U nit Test/S QA Sprint Testing
	611	MAP2 HPES Developers and MAP2 HPES SQA T esters
	612	Complete S print 4 Us er Story/U nit Test/S QA Sprint Testing
	613	MAP2 HPES Developers and MAP2 HPES SQA T esters
	614	Complete S print 5 Us er Story/U nit Test/S QA Sprint Testing
	615	MAP2 HPES Developers and MAP2 HPES SQA T esters
	616	Complete S ystem Test ing
	617	MAP2 HPES SQA Test T eam
	618	Complete C ompliance Reviews, V &V Testing , IOC
	619	VA Complia nce Bodies and V&V T est Team
	620	Test Envir onments
	621	A test env ironment i s an envir onment con taining ha rdware or virtual ma chines, si mulators, software t ools, and other supp ort elemen ts needed to conduct a test. S imulators in a servi ce-oriente d architec ture, such as used i n VA, incl ude "mock services," which sim ulate the behavior o f VA servi ces. The M AE contain s mock ser vices in s ome areas to control the inter faces and data retri eved and s imulate te st scenari os. There are multip le test en vironments within th e MAE to s upport dif ferent lev els of tes ting.
	622	Test Envir onment Con figuration s
	623	The MAE in cludes the following environme nts and to ols that c an be util ized by th e MAP2 Pro gram team. Developme nt environ ment – thi s includes the follo wing servi ces and to ols:
	624	Git/Stash – code rep ositories are provid ed for the mobile ap plication as well as access to the Mobil e Adapter (Health Ad apter) cod e reposito ry to prov ide the de velopment team acces s to the c ommon serv ices provi ded
	625	Jenkins – for buildi ng the sof tware into deployabl e units
	626	Fortify – for scanni ng the sof tware
	627	JIRA – for executing the agile processes , document ing user s tories, cr eating agi le boards, documenti ng softwar e issues, etc. Compl iance issu es are inc luded in t he JIRA pr oject so t he develop ment team can track completion of these requiremen ts and att ach the ap propriate documents.
	628	Confluence – for doc umenting t he mobile applicatio n in a wik i format. The wiki c an be used to delive r the docu ments need ed by MAP2 and the V &V team
	629	Developmen t integrat ion server s – to inc lude mock services f or standar d VA enter prise serv ices to be called by the Mobil e Adapter services a nd/or dire ctly from mobile app lications
	630	Developmen t – develo pers use t his enviro nment for internal c omponent i ntegration not exter nal integr ation. It allows the users to test deplo yment of m ultiple de velopers' code from the code r epository.
	631	Features o f this env ironment a re:
	632	Mock Servi ces for ex ternal sys tems, such as Identi ty and Acc ess Manage ment (IAM) , Master V eteran Ind ex (MVI), Administra tive Data Repository (ADR), Vi stA (or de v VistA), Medical Do main Web S ervices (M DWS), Corp orate Data Warehouse (CDW), et c.
	633	All test d ata and fa ke patient s/users
	634	Developmen t-Demo – H PES SQA Te sters assi gned to th e Scrum te am will pe rform feat ure accept ance testi ng of user stories i n this env ironment a s well. De velopers u se this en vironment to deploy applicatio ns before they are d elivered t o the busi ness spons or for the following :
	635	Allows use r communit ies to use and comme nt on appl ications i n developm ent
	636	Allows for testing o f applicat ions and s ervices fr om non-GFE devices
	637	Demonstrat e code to SMEs durin g sprint r eviews
	638	Allow for collaborat ion with e xternal ag encies or partners
	639	Features o f this env ironment a re:
	640	An isolate d environm ent so it can provid e Public F acing IP a ddresses
	641	Mock Servi ces for ex ternal sys tems, such as IAM, M VI, ADR, V istA (or d ev VistA), MDWS, CDW , etc.
	642	All test d ata and fa ke patient s/users
	643	Quickly re -deploy sp ecific app lications and servic es based o n demonstr ation need s regardle ss of wher e they are in the De velopment cycle
	644	Developers will use this envir onment to perform co mponent-in tegration testing an d integrat e with som e VA servi ces prior to system test. Deve lopment-Te st – HPES SQA Tester s will per form syste m testing in this en vironment.
	645	Features o f this env ironment a re:
	646	Real Test Services f or externa l systems, such as I AM, MVI, A DR, VistA (or dev Vi stA), MDWS , CDW, etc .
	647	Mock Servi ces used w here devel opment int egrations are not al lowed
	648	All test d ata and fa ke patient s/users
	649	Integratio n – This e nvironment will be u sed for sy stem testi ng and V&V and Compl iance test ing. When more resou rces are o btained, t en this wi ll be a sy stem testi ng environ ment.
	650	Features o f this env ironment a re:
	651	Real Test Services f or externa l systems, such as I AM, MVI, A DR, VistA (or dev Vi stA), MDWS , CDW, etc .
	652	Mock Servi ces used w here neede d, such as CDW
	653	All test d ata and fa ke patient s/users
	654	PerfTest – Performan ce testing will occu r here whe n this is establishe d so that real data is not use d in test scripts an d results capture. F eatures of this envi ronment ar e:
	655	Mock Servi ces for ex ternal sys tems, such as IAM, M VI, ADR, V istA (or d ev VistA), MDWS, CDW , etc.
	656	Mock Servi ces used w here neede d, such as CDW
	657	All test d ata and fa ke patient s/users
	658	V&V (futur e) – In th e future, this envir onment wil l be used for V&V an d Complian ce testing . Since V& V is essen tially a p re-complia nce check, then this environme nt is call ed "Comp-T est" as sh orthand.
	659	Features o f this env ironment a re:
	660	Real Test Services f or externa l systems, such as I AM, MVI, A DR, VistA (or dev Vi stA), MDWS , CDW, etc .
	661	Mock Servi ces used w here neede d, such as CDW
	662	All test d ata and fa ke patient s/users
	663	Pre-Produc tion – Pri mary purpo ses:
	664	Installati on of the final pack age
	665	Recreation of produc tion issue s
	666	Features o f this env ironment a re:
	667	Real Pre-P rod Servic es for ext ernal syst ems connec ted to exi sting pre- prod envir onments th at have pr oduction-l ike PII/PH I data
	668	Copy of Pr oduction D ata
	669	Secure env ironment i n the "Hig h" system boundary o f the VAMF
	670	This envir onment is not open t o the Inte rnet.
	671	Access to these MAE environmen ts is obta ined throu gh the Web and Mobil e Solution request p rocess. Th e MAP2 HPE S Developm ent and MA P2 HPES SQ A Test tea ms only ha ve access to the MAE Developme nt environ ments. The MAP2 CM m anager mus t have acc ess to all MAE devel opment env ironments for auditi ng purpose s. The MAE Maintenan ce Team ma intains al l of the M AE environ ments, and this team should no t include any develo pers from the MAP2 P rogram tea m.
	672	Base Syste m Hardware
	673	Note: Addi tional inf ormation f or the Bas e System H ardware se ction will be added/ updated in future it erations a s needed.
	674	Table 7 sh ows the sy stem resou rces for t he test ef fort prese nted in th is Master Test Plan.
	675	The specif ic element s of the t est system may not b e fully un derstood i n early it erations; so, this s ection may be comple ted over t ime. The t est system should si mulate the productio n environm ent as clo sely as po ssible, sc aling down the concu rrent acce ss and dat abase size , and so f orth, if a nd where a ppropriate .
	676	Table 7: S ystem Hard ware Resou rces
	677	Resource
	678	Quantity
	679	Name and T ype
	680	Database S erver
	681
	682	TBD
	683	Network or Subnet
	684
	685	TBD
	686	Server Nam e
	687
	688	TBD
	689	Database N ame
	690
	691	TBD
	692	Client Tes t PCs
	693
	694	TBD
	695	Special Co nfiguratio n Requirem ents
	696
	697	TBD
	698	Test Repos itory
	699
	700	TBD
	701	Network or Subnet
	702
	703	TBD
	704	Server Nam e
	705
	706	TBD
	707	Test Devel opment Com puters (PC s)
	708
	709	TBD
	710
	711	Base Softw are Elemen ts in the Test Envir onments
	712	Note: Addi tional inf ormation f or the Use r Function ality Test section w ill be add ed/updated in future iteration s as neede d.Table 8 describes the base s oftware el ements tha t are requ ired in th e test env ironment f or this Ma ster Test Plan.Table 8: Softwa re Element s
	713	Software E lement Nam e
	714	Version
	715	Type and O ther Notes
	716	Windows
	717	7.0
	718	Desktop Op erating Sy stem
	719	IOS
	720	TBD
	721	Apple Oper ating Syst em
	722	Android
	723	TBD
	724	Android Op erating Sy stem
	725	Internet E xplorer
	726	11
	727	Laptop Int ernet Brow ser
	728	Chrome
	729	TBD
	730	Mobile Int ernet Brow ser
	731	Chrome
	732	TBD
	733	Desktop In ternet Bro wser
	734	Safari
	735	TBD
	736	Mobile Int ernet Brow ser
	737	Safari
	738	TBD
	739	Desktop In ternet Bro wser
	740	Staffing and Traini ng Needs
	741	Table 9 de scribes th e HPES per sonnel res ources nee ded to pla n, prepare , and exec ute this M aster Test Plan. Thi s document does not apply to a ny VA user training needed.
	742	Table 9: H PES Staffi ng Resourc es
	743	Testing Ta sk
	744	Quantity o f Personne l Needed
	745	VA MA Deve lopment Ph ase
	746	Duration/ Days
	747	Create the Master Te st Plan – Program-Le vel
	748	1
	749	Planning
	750	TBD
	751	Create/Upd ate Master Test Plan – Applica tion-Level
	752	1
	753	Planning/D evelopment
	754	TBD
	755	Establish the Test E nvironment
	756	1
	757	Planning/D evelopment
	758	TBD
	759	Create/Upd ate Test C ases and S cripts
	760	5
	761	Developmen t
	762	TBD
	763	Perform Fe ature Veri fication/S ystem Test s
	764	5
	765	Developmen t
	766	TBD
	767	Test Manag ement Repo rting
	768	1
	769	Developmen t
	770	TBD
	771	Coordinate with V&V Test Team and Compli ance Bodie s
	772	3
	773	Developmen t/Complian ce Review
	774	TBD
	775	The MAP2 P rogram tea m is respo nsible for training their deve lopers and test team members o n the tech nologies i nvolved. T he MAP2 Pr ogram team will be r esponsible for train ing on:
	776	The MAE Ch ange Contr ol Board ( CCB) proce sses regar ding the u pdates of environmen ts.
	777	JIRA and w iki traini ng for pro gram perso nnel using these too ls.
	778	Risks and Constraint s
	779	Risks are tracked in JIRA both at the MA P2 mobile applicatio n level an d the MAP2 program l evel. This section w ill be upd ated as ri sks and co nstraints are identi fied. Test ing relate d risks wi ll be iden tified by the HPES S QA Test Ma nager and submitted to the MAP 2 HPES Pro gram Manag er for inc lusion on the MAP2 R isk Regist er.
	780	Again, ris ks are tra cked in JI RA both at the mobil e applicat ion level and the MA P program level. Ris ks that af fect the c ompletenes s or effec tiveness a re the fol lowing:
	781	Lack of En vironments – need to ensure VA will be c onfiguring the MAE I ntegration and Pre-P roduction environmen t to match the confi guration o f the MAE Developmen t-Test env ironment. This is to fully sup port integ rated syst em testing , V&V test ing, and p erformance testing. If the con figuration is not th e same, th e overall test appro ach is com promised.
	782	If an inte grated MAE Developme nt-Test (i .e., CDW, HDR, Vista , etc.) en vironment is not ava ilable, th en there i s a risk t hat testin g will not be adequa te for VA V&V and pr oduction r elease
	783	Assuming p erformance and load testing wi ll be perf ormed in t he MAE Dev elopment-T est enviro nment, and this envi ronment is not a clo se match t o the prod uction env ironment, then then there is a higher ri sk that th e migrated applicati on will no t perform as expecte d in the P roduction environmen t
	784	In the cas e of read- only appli cations, t he pre-pro duction en vironment can be int egrated wi th other p re-product ion/produc tion envir onments fo r UFT test ing, but o bviously a pre-produ ction envi ronment ca nnot write to a prod uction env ironment. This needs to be man aged close ly on an a pplication by applic ation basi s and a me thod of te sting cons tructed su ch that it is clear that adequ ate testin g has been performed .
	785	Test Metri cs
	786	Metrics us ed in test ing provid e an objec tive and m easurable gauge of t he coverag e and prog ress of th e test eff ort as wel l as the q uality of the softwa re itself. Table 10 displays t he test me trics that will be c aptured an d provided throughou t the MA T est life c ycle.Table 10: Test Metrics
	787	Report Nam e
	788	Report Typ e
	789	Report Des cription
	790	Projected vs. Actual Test Effo rt
	791	Project Ma nagement
	792	The Estima ted vs. Ac tual Test Effort rep ort provid es managem ent with a view of h ow well th e project met its ag reed upon delivery t imeline de termined a t project initiation . The data used to c onstruct t his indica tor is as follows:
	793	Estimated Hours: est imated hou rly effort associate d with com pleting al l activiti es require d to meet the exit c riteria fo r a given test phase
	794	Actual Hou rs: hourly effort as sociated w ith comple ting all a ctivities required t o meet the exit crit eria for a given tes t phase
	795	Actual Tes t Effort b y Level
	796	Project Ma nagement
	797	The Actual Test Effo rt by leve l report p rovides a breakdown of the tes ting effor t by each level for each appli cation in a MA relea se.
	798	Requiremen t Test Cas e Coverage
	799	Coverage
	800	Requiremen ts Test Ca se Coverag e report p rovides a method to measure th e percenta ge of cove rage throu gh test pr ocedures. The data u sed to con struct thi s indicato r is as fo llows:
	801	Baseline R equirement s: the num ber of req uirements approved b y the team and clien t to be ad dressed in the MA re lease, ite ration, or build und er develop ment
	802	Requiremen ts Mapped to Test Sc ripts: the number of baseline requiremen ts that ha ve been ma pped to te st scripts for the s pecified r elease
	803	Overall Te st Case/Sc ript Execu tion Statu s
	804	Progressio n
	805	The Overal l Test Pro cedure Exe cution Sta tus Report details t he progres sion and e xtent of t he test ef fort at a given poin t and time and the p ercentage of test ca ses execut ed in addi tion to:
	806	Total Test Scripts – the numbe r of test scripts sc heduled to be run in the curre nt test cy cle. The t est cycle will be de noted in t he slide
	807	Scripts to be Execut ed – the n umber of s cripts yet to be run
	808	Currently Passed – t he number of scripts for the c urrent cyc le that ha ve passed
	809	First Time Passed – the number of "Curre ntly Passe d" scripts that pass ed the fir st time th ey were ex ecuted
	810	Daily Test Execution Status
	811	Progressio n
	812	The Daily Test Execu tion Statu s reports details th e daily pr ogression of the tes ting effor t versus t he planned progressi on by refl ecting how many scri pts have p assed and failed on a daily ba sis. The r eport capt ures the f ollowing m etrics:
	813	Test Scrip ts in Rele ase: numbe r of scrip ts that pr ovide cove rage for t he release
	814	Test Scrip ts Passed Cumulative Planned: number of scripts th e test tea m expects to have pa ssed at a given poin t in time
	815	Test Scrip ts Passed Cumulative Actual: n umber of s cripts tha t have act ually pass ed at a gi ven point in time
	816	Test Scrip ts Passed Daily: num ber of scr ipts run i n a given day that h ave succes sfully pas sed
	817	Test Scrip ts Failed Daily: num ber of scr ipts run i n a given day that f ailed
	818	Open Defec ts by Seve rity
	819	Progressio n
	820	Open Defec ts by Seve rity Repor t captures the ratio of report ed defects by the se verity lev el and the :
	821	Total numb er of defe cts identi fied
	822	Number of defects as sociated t o test cas e/test scr ipt and us er stories /requireme nts
	823	Percentage of defect s listed b y cause an d severity
	824	Attachment A – Defec t Level De finitions
	825	Level 1 Cr itical - T he defect results in the failu re of the complete s oftware sy stem, of a subsystem , or of a software u nit (progr am or modu le) within the syste m.
	826	Any defect that comp romises pa tient safe ty or syst em securit y. Example s of syste m security defects i nclude bre ach of con fidentiali ty require ments of t he Privacy Act, the Health Ins urance Por tability a nd Account ability Ac t (HIPAA), or Federa l Tax Info rmation gu idelines.
	827	Loss of sy stem funct ionality c ritical to user oper ations wit h no suita ble workar ound, i.e. , there is no way to achieve t he expecte d results using the applicatio n
	828	System cra sh or hang that prev ents furth er testing or operat ion of the complete applicatio n or a sec tion of th e applicat ion
	829	Any defect that caus es corrupt ion of dat a from a r esult of t he system (as oppose d to user error)
	830	Any defect in which inappropri ate transm issions ar e consiste ntly gener ated or ap propriate transmissi ons of HL7 messages fail to be generated
	831	Loss of fu nctionalit y resultin g in erron eous eligi bility/enr ollment de terminatio ns or comm unications not being sent
	832	Level 2 Hi gh- The de fect resul ts in the failure of the compl ete softwa re system, of a subs ystem, or of a softw are unit ( program or module) w ithin the system. Th ere is no way to mak e the fail ed compone nt(s) func tion. Howe ver, there are accep table proc essing alt ernatives which will yield the desired r esult.
	833	A major de fect in th e function ality that does not result in corruption of data
	834	A major de fect in th e function ality resu lting in a failure o f all or p art of the applicati on, where:
	835	The expect ed results can tempo rarily be achieved b y alternat e means. T he custome r indicate s the work around is acceptabl e for the short term .
	836	Any defect that does not confo rm to Sect ion 508 st andards
	837	Any defect that resu lts in ina ccurate or missing r equirement s
	838	Any defect that resu lts in inv alid authe ntication or authent ication of an invali d end user
	839	Level 3 Me dium - The defect do es not res ult in a f ailure, bu t causes t he system to produce incorrect , incomple te, or inc onsistent results, o r the defe ct impairs the syste ms usabili ty.
	840	Minor func tionality is not wor king as in tended and a workaro und exists but is no t suitable for long term use
	841	The inabil ity of a v alid user to access the system consisten t with gra nted privi leges
	842	Typographi cal or gra mmatical e rrors in t he applica tion, incl uding inst allation g uides, use r guides, training m anuals, an d design d ocuments
	843	Any defect producing cryptic, incorrect, or inappr opriate er ror messag es
	844	Any defect that resu lts from t he use of non-standa rd data te rminology in the app lication o r document ation, as defined by the Depar tment of V eterans Af fairs
	845	Cosmetic i ssues that are impor tant to th e integrit y of the p roduct, bu t do not r esult in d ata entry and or dat a quality problems
	846	Level 4 Lo w - The de fect does not cause a failure, does not impair usa bility, an d the desi red proces sing resul ts are eas ily obtain ed by work ing around the defec t.
	847	Minor loss of, or de fect in th e function ality wher e a long t erm use ex ists
	848	Low-level cosmetic i ssues
	849	Attachment B – Test Case/Scrip t – Covera ge
	850	The exampl e below ca n be used by the MAP 2 program team membe rs to buil d test cas e/scripts, or the Te st Case/Sc ript templ ate can be added to this secti on of the MTP.Test C ase: Descr iption TC Pre Condit ions:
	851	Pre-Condit ions
	852	Post-Condi tions
	853	Acceptance Criteria
	854	The user/t ester shou ld have ac cess to al l needed a reas, such as the st ore, to do wnload and install t he app. Th e user/tes ter should have the app instal led on the device. T he user/te ster shoul d be famil iar with M obile apps controls, such as s lides and spinners.
	855
	856
	857	Test Scrip t:Test Scr ipt Descri ption: Tes t Procedur e
	858	Pre-Condit ions
	859	Post-Condi tions
	860	Acceptance Criteria
	861
	862
	863
	864	Auto Imple mentation: Manual Im plementati on: Test I nputs: Tes t Design
	865	Step
	866	Type
	867	Note
	868	Descriptio n
	869	1
	870	Step
	871
	872
	873	2
	874	VP
	875	.
	876
	877	3
	878	Step
	879
	880
	881	4
	882	VP
	883
	884