3. EPMO Open Source Coordination Office Redaction File Detail Report

Produced by Araxis Merge on 10/5/2017 9:24:06 AM Central Daylight Time. See www.araxis.com for information about Merge. This report uses XHTML and CSS2, and is best viewed with a modern standards-compliant browser. For optimum results when printing this report, use landscape orientation and enable printing of background images and colours in your browser.

3.1 Files compared

#	Location	File	Last Modified
1	RT_2_47.zip\docs	EPIP_Test_Evaluation_(RT_2.0_47).doc	Thu Oct 5 12:54:44 2017 UTC
2	RT_2_47.zip\docs	EPIP_Test_Evaluation_(RT_2.0_47).doc	Thu Oct 5 13:24:04 2017 UTC

3.2 Comparison summary

Description	Between Files 1 and 2
Description	Text Blocks	Lines
Unchanged	4	542
Changed	0	0
Inserted	0	0
Removed	3	3

3.3 Comparison options

Whitespace
Character case	Differences in character case are significant
Line endings	Differences in line endings (`CR` and `LF` characters) are ignored
CR/LF characters	Not shown in the comparison detail

3.4 Active regular expressions

No regular expressions were active.

3.5 Comparison detail

	1	Test Evalu ation Temp lateExisti ng Product Intake Pr ogram (EPI P)
	2	Patch RT2 .047
	3	Test Evalu ation
	4
	5	Department of Vetera ns Affairs
	6	August 201 7
	7	Version 1. 0
	8	Revision H istory
	9	Note: The revision h istory cyc le begins once chang es or enha ncements a re request ed after t he Communi cations Pl an has bee n baseline d.
	10	DateVersio nDescripti onAuthor08 /07/20171. 0Initial d ocument.EP IP Project TeamArtif act Ration ale
	11	The test e valuation document i s the prim ary output of the te st and eva luation pr ocess, an integral p art of the systems e ngineering process, which iden tifies lev els of per formance a nd assists the devel oper in co rrecting d eficiencie s.
	12	Table of C ontents
	13	11.
	14	Test Evalu ation Intr oduction
	15
	16
	17	11.1.
	18	Test Evalu ation Scop e
	19
	20
	21	11.2.
	22	Test Archi tecture
	23
	24
	25	11.3.
	26	Test Envir onment/Con figuration
	27
	28
	29	21.4.
	30	Installati on Process
	31
	32
	33	22.
	34	Test Data
	35
	36
	37	23.
	38	Issues
	39
	40
	41	24.
	42	Test Execu tion Log
	43
	44
	45	35.
	46	Test Defec t Log
	47
	48
	49	36.
	50	Test Resul ts Summary
	51
	52
	53	36.1.
	54	Defect Sev erity and Priority L evels
	55
	56
	57	36.2.
	58	Total Defe cts by Sev erity Leve l
	59
	60
	61	36.3.
	62	Breakdown of Test Re sults
	63
	64
	65	36.4.
	66	Performanc e Testing
	67
	68
	69	37.
	70	Test Cover age
	71
	72
	73	37.1.
	74	Requiremen ts Covered
	75
	76
	77	47.2.
	78	Section 50 8 Complian ce Coverag e
	79
	80
	81	48.
	82	Suggested Actions
	83
	84
	85	49.
	86	Defect Sev erity and Priority D efinitions
	87
	88
	89	59.1.
	90	Defect Sev erity Leve l
	91
	92
	93	59.1.1.
	94	Severity L evel 1 – C ritical
	95
	96
	97	59.1.2.
	98	Severity L evel 2 - H igh
	99
	100
	101	69.1.3.
	102	Severity L evel 3 - M edium
	103
	104
	105	69.1.4.
	106	Severity L evel 4 - L ow
	107
	108
	109	69.2.
	110	Priority C lassificat ions
	111
	112
	113	69.2.1.
	114	Priority 1 - Resolve Immediate ly
	115
	116
	117	69.2.2.
	118	Priority 2 - Give Hi gh Attenti on
	119
	120
	121	69.2.3.
	122	Priority 3 - Normal Queue
	123
	124
	125	69.2.4.
	126	Priority 4 - Low Pri ority
	127
	128
	129	710.
	130	Optional T ables, Cha rts, and G raphs
	131
	132
	133	811.
	134	Document A pproval Si gnatures
	135
	136
	137	9Appendix A - Test E xecution L og
	138
	139
	140	10Appendix B – Defec t Log
	141
	142
	143
	144
	145	Test Evalu ation Intr oduction
	146	The purpos e of this Test Evalu ation is t o:
	147	Identify t he testing approach used.
	148	Present a summary an alysis of the key te st results from the remediatio n of this intake for review an d assessme nt by desi gnated sta keholders.
	149	Provide a general st atement of the quali ty of the system und er test.
	150	Make recom mendations for futur e testing efforts.
	151	Test Evalu ation Scop e
	152	The scope of this Te st Evaluat ion is to verify the functiona lity of th e Patch RT 2.047 co de modific ation, as determined by Functi onal, Comp onent Inte gration/Sy stem, and Regression testing. Testing ac tivities f ollowed th e specific ations out lined in t he followi ng Master Test Plan: RT2.047 Master Te st Plan (i ncluded in Appendix A).
	153	Test Archi tecture
	154	Following are the EP IP test ac counts use d by the L eidos Deve lopment an d SQA Test ing teams to test RT 2.047.
	155	Developmen t Test Acc ounts
	156	(For Unit Testing)SQ A Test Acc ounts
	157	(For Funct ional, Reg ression, a nd Compone nt Integra tion and S ystem Test ing)VistAS 1 ( altern ate name: D1S1)VistA G1 (altern ate name: D1G1) Vist AS2 (alter nate name: D1S2) –
	158	for CPRS G UI testing onlyVistA G2 (altern ate name: D1G2) –
	159	for CPRS G UI testing onlyTest Environmen t/Configur ation
	160	The EPIP t est accoun ts are mai ntained by the EPIP System Adm inistrator , who inst alls all V A-released patches a s soon as they are n ationally released. All EPIP t est accoun ts are clo ned from e xisting VA Enterpris e Testing Services ( ETS) test accounts. The Comput erized Pat ient Recor d System ( CPRS) Grap hical User Interface (GUI) exe cutable is configure d for each VistA ins tance util izing a un ique Inter net Protoc ol (IP) ad dress to c onnect to the VistA applicatio ns. Any up dates to t he CPRS GU I executab le are han dled by th e EPIP Sys tem Admini strator.
	161	All EPIP T est Engine ers and De velopers w ho have th e proper c redentials can acces s the test accounts. The VA Au stin Infor mation Tec hnology Ce nter (AITC ) support team reset s password s and sets up new ac cess crede ntials on an as-need ed basis.
	162	Installati on Process
	163	As soon as the remed iation pro cess is co mplete and the patch is availa ble for te sting, a K IDS build is created in the De velopment account an d then sen t to FORUM for final packaging . The patc h is then submitted to the VA SQA Lead’s Mailman a ccount for installat ion.
	164	An EPIP De veloper or Test Engi neer utili zes the KI DS Install ation proc ess to ext ract the b uild from the patch and instal l the buil d into a t est accoun t. The ind ividual wh o installs the patch verifies the routin e checksum s and also checks fo r errors d uring the installati on process . If the p atch is su ccessfully installed without a ny errors, then the EPIP Test team proce eds with F unctional, Regressio n, and Com ponent Int egration a nd System testing. I f defects are found, then the Developmen t team wor ks to find a resolut ion and cr eates new versions o f the patc h until al l defects are resolv ed.
	165	Test Data
	166	The SQA Te sting team utilizes the test d ata in the designate d test acc ounts (D1G 1, D1G2).
	167	The test d ata is enc rypted fol lowing the standards set forth by the VA Office of Informati on & Techn ology (OI& T). All Pe rsonally I dentifiabl e Informat ion (PII) and Protec ted Health Informati on (PHI) i s scrubbed and is no t availabl e to the T est Engine ers.
	168	All testin g is execu ted using encrypted test patie nts availa ble from a ny of the EPIP test accounts. Examples o f encrypte d test pat ients:
	169	AAAHURMMX, XPHY
	170	BADHB, HAA DXS
	171	FDHUX, YHI J
	172	All tests were execu ted manual ly by EPIP Test Engi neers.
	173	Issues
	174	The follow ing issues were enco untered du ring testi ng of RT2 .047.
	175	TitleIssue Descripti onTypeSeve rityN/AN/A N/AN/ATest Execution Log
	176	The Test E xecution L og records the execu tion of te st scripts and docum ents the t est result s for each test scri pt.
	177	The SQA Te sting team utilizes the Ration al Quality Managemen t (QM) too l for all testing ac tivities. All test d ocuments a re stored in the EPI P reposito ry, includ ing the Ma ster Test Plan, Test Suites, T est Cases, and Test Scripts. T est execut ion is per formed, an d test res ults recor ded, in Ra tional QM. The Test Engineer a dds the te st results to the Te st Executi on records to indica te whether testing a chieved Pa ss or Fail status.
	178	The Test E xecution r ecords for RT2.047 are inclu ded in the EPIP Patc h RT2.04 7 Master T est Plan. The Master Test Plan is availa ble in App endix A.
	179	Test Defec t Log
	180	The Test D efect Log is a tool for record ing, analy zing, trac king, and documentin g the clos ure of def ects. It s pecifies t he screen, field, be havior or result tha t occurred , and the IEEE-defin ed Severit y Level. I t includes enough in formation for the de veloper to find and re-create the defect . The Defe ct Log is available in Appendi x B.
	181	Test Resul ts Summary
	182	SQA testin g for this intake st arted in t he Dev1 Go ld1 test e nvironment on June 2 6, 2017 an d ended on July 14, 2017.
	183	Test versi on 1 was i nstalled i n the test environme nt on June 26, 2017 after Unit testing i n Dev1 Sil ver1 was c ompleted. Upon compl etion of I ntegration testing ( Component Integratio n and Syst em Testing , Function al Testing , and Regr ession Tes ting), (0) defects w ere found and report ed.
	184	Defect Sev erity and Priority L evels
	185	A defect i s defined as a flaw in a compo nent or sy stem that can cause the compon ent or sys tem to fai l to perfo rm its req uired func tion, e.g. , an incor rect state ment or da ta definit ion. A def ect, if en countered during exe cution, ma y cause a failure of the compo nent or sy stem.
	186	Defects ar e categori zed accord ing to sev erity and priority l evels. The test anal yst assign s the seve rity, whil e the deve lopment ma nager assi gns the pr iority for repair. F or more in formation, see Defec t Severity and Prior ity Defini tion in th is Test Ev aluation.
	187	Total Defe cts by Sev erity Leve l
	188	The Defect Log in Ap pendix B d isplays th e defects encountere d while te sting this patch, an d the seve rity level of each.
	189	Breakdown of Test Re sults
	190	Testing wa s complete d on July 14, 2017. All test r esults wer e recorded in Ration al QM. Det ailed resu lts are av ailable in the EPIP Patch RT2 .047 Mast er Test Pl an (see Ap pendix A).
	191	Performanc e Testing
	192	Performanc e testing was not co nducted.
	193	Test Cover age
	194	The EPIP P atch RT2. 047 Maste r Test Pla n contains details o n test cov erage (see Appendix A).
	195	Requiremen ts Covered
	196	The requir ements for RT2.047 are store d in the R ational Re quirements Managemen t (RM) app lication. The test c ases store d in Ratio nal Qualit y Manageme nt (QM) ar e used to validate t hat the re quirements have been addressed , providin g full tra ceability. The user stories st ored in Ra tional Con figuration Managemen t (CM) are linked to the requi rements in RM and te st cases i n QM.
	197	The follow ing links provide ac cess to th e various Record_Tra cking repo sitories i n the Rati onal toolk it. If lin k translat ion issues prevent d irect acce ss, copy a nd paste t he URLs in to your br owser.
	198	Record Tra cking (RM) – Go to A rtifacts, then Brows e Artifact s. Locate the EPIP f older on t he left si de of the page and e xpand it t o display patch fold ers. Each patch fold er contain s the requ irements f or the pat ch number shown in t he folder name.

	199	Record Tra cking (QM) – Go to P lanning, t hen Browse Test Plan s, and the n search f or the Mas ter Test P lan you ne ed. The Ma ster Test Plan and t est cases are linked to requir ements.

	200	Record Tra cking (CM) – Go to P lans, then All Plans , and then search fo r the Spri nt Plan yo u need. Th e user sto ries in ea ch Plan ar e linked t o requirem ents and t est cases.

	201	Section 50 8 Complian ce Coverag e
	202	Section 50 8 test res ults will be reporte d to VA in the follo wing docum ents:
	203	EPIP_VASec tion508_Co mpliance_T est_Result s_(RT_2.0_ 47)
	204	EPIP_VASec tion508_Co nformance_ Validation _Statement _(RT_2.0_4 7)
	205	EPIP_VASec tion508_Ve rifiable_O bjective_E vidence_(R T_2.0_47)
	206	Suggested Actions
	207	Leidos rec ommends mo ving this patch to I OC testing .
	208	Defect Sev erity and Priority D efinitions
	209	The classi fication o f defects within a s ystem exam ines both the severi ty and pri ority of t he defect.
	210	Severity i s a measur e of how g reat the i mpact is o n the user ’s ability to comple te the doc umented ac tions with in the sys tem.
	211	Priority d etermines the speed with which a given d efect must be repair ed.
	212	Defect cla ssificatio n may be d etermined either bec ause testi ng is dela yed by a f ailure in the system or becaus e a cumber some worka round prev ents a use r from com pleting th e assigned tasks. Bo th severit y and prio rity measu res must b e recorded when sche duling def ect resolu tion tasks .
	213	Defect Sev erity Leve l
	214	The follow ing subsec tions iden tify the d efect seve rity level s.
	215	Severity L evel 1 – C ritical
	216	Institute of Electri cal and El ectronics Engineers (IEEE) def inition: T he defect results in the failu re of the complete s oftware sy stem, of a subsystem , or of a software u nit (progr am or modu le) within the syste m.
	217	Any defect that comp romises pa tient safe ty or syst em securit y. Example s of syste m security defects i nclude bre ach of con fidentiali ty require ments of t he Privacy Act, the Health Ins urance Por tability a nd Account ability Ac t (HIPAA), or Federa l Tax Info rmation gu idelines.
	218	Loss of sy stem funct ionality c ritical to user oper ations wit h no suita ble workar ound, i.e. , there is no way to achieve t he expecte d results using the applicatio n.
	219	System cra sh or hang that prev ents furth er testing or operat ion of the complete applicatio n or a sec tion of th e applicat ion.
	220	Any defect that caus es corrupt ion of dat a from a r esult of t he system (as oppose d to user error).
	221	Any defect in which inappropri ate transm issions ar e consiste ntly gener ated or ap propriate transmissi ons of HL7 messages fail to be generated .
	222	Loss of fu nctionalit y resultin g in erron eous eligi bility/enr ollment de terminatio ns or comm unications not being sent.
	223	Severity L evel 2 - H igh
	224	IEEE defin ition: The defect re sults in t he failure of the co mplete sof tware syst em, of a s ubsystem, or of a so ftware uni t (program or module ) within t he system. There is no way to make the f ailed comp onent(s) f unction. H owever, th ere are ac ceptable p rocessing alternativ es which w ill yield the desire d result.
	225	A major de fect in th e function ality that does not result in corruption of data.
	226	A major de fect in th e function ality resu lting in a failure o f all or p art of the applicati on, where:
	227	The expect ed results can tempo rarily be achieved b y alternat e means. T he custome r indicate s the work around is acceptabl e for the short term .
	228	Any defect that does not confo rm to Sect ion 508 st andards.
	229	Any defect that resu lts in ina ccurate or missing r equirement s.
	230	Any defect that resu lts in inv alid authe ntication or authent ication of an invali d end user .
	231	Severity L evel 3 - M edium
	232	IEEE defin ition: The defect do es not res ult in a f ailure, bu t causes t he system to produce incorrect , incomple te, or inc onsistent results, o r the defe ct impairs the syste ms usabili ty.
	233	Minor func tionality is not wor king as in tended and a workaro und exists but is no t suitable for long term use
	234	The inabil ity of a v alid user to access the system consisten t with gra nted privi leges
	235	Typographi cal or gra mmatical e rrors in t he applica tion, incl uding inst allation g uides, use r guides, training m anuals, an d design d ocuments
	236	Any defect producing cryptic, incorrect, or inappr opriate er ror messag es
	237	Any defect that resu lts from t he use of non-standa rd data te rminology in the app lication o r document ation, as defined by the Depar tment of V eterans Af fairs
	238	Cosmetic i ssues that are impor tant to th e integrit y of the p roduct, bu t do not r esult in d ata entry and or dat a quality problems.
	239	Severity L evel 4 - L ow
	240	IEEE defin ition: The defect do es not cau se a failu re, does n ot impair usability, and the d esired pro cessing re sults are easily obt ained by w orking aro und the de fect.
	241	Minor loss of, or de fect in th e function ality wher e a long t erm use ex ists
	242	Low-level cosmetic i ssues.
	243	Priority C lassificat ions
	244	The follow ing subsec tions iden tify the a ppropriate actions f or defects at each p riority le vel, per d efinitions of IEEE.
	245	Priority 1 - Resolve Immediate ly
	246	Further de velopment and/or tes ting canno t occur un til the de fect has b een repair ed. The sy stem canno t be used until the repair has been affe cted.
	247	Priority 2 - Give Hi gh Attenti on
	248	The defect must be r esolved as soon as p ossible be cause it i s impairin g developm ent and/or testing a ctivities. System us e will be severely a ffected un til the de fect is fi xed.
	249	Priority 3 - Normal Queue
	250	The defect should be resolved in the nor mal course of develo pment acti vities. It can wait until a ne w build or version i s created.
	251	Priority 4 - Low Pri ority
	252	The defect is an irr itant that should be repaired, but can b e repaired after mor e serious defects ha ve been fi xed.
	253	Optional T ables, Cha rts, and G raphs
	254	None.
	255	Document A pproval Si gnatures
	256	Signed: __ __________ __________ __________ __________ __________ __________ _________
	257	Program/Pr oject Mana ger
	258	Date
	259	Signed: __ __________ __________ __________ __________ __________ __________ _________
	260	Business S ponsor Rep resentativ e
	261	Date
	262	Signed: __ __________ __________ __________ __________ __________ __________ _________
	263	Test Lead
	264	Date
	265	Appendix A - Test Ex ecution Lo g
	266	The Test E xecution R ecords for RT2.047 are inclu ded in the EPIP Patc h RT2.04 7 Master T est Plan.
	267
	268	Appendix B – Defect Log
	269	No defects were foun d during t esting of RT2.047.
	270	Defect IDA ffected Sc reenAffect ed FieldOb served Beh aviorSever ityDescrip tionN/AN/A N/AN/AN/AN o defects were found during Un it Testing of versio n 1.0.N/AN /AN/AN/AN/ ANo defect s were fou nd during Component Integratio n/Systems Testing of version 1 .0.N/AN/AN /AN/AN/ANo defects w ere found during Reg ression te sting of v ersion 1.0 .N/AN/AN/A N/AN/ANo d efects wer e found du ring Funct ional Test ing of ver sion 1.0.
	271	_156869522 5.pdf