2. EPMO Open Source Coordination Office Redaction File Detail Report

Produced by Araxis Merge on 10/17/2017 1:23:34 PM Central Daylight Time. See www.araxis.com for information about Merge. This report uses XHTML and CSS2, and is best viewed with a modern standards-compliant browser. For optimum results when printing this report, use landscape orientation and enable printing of background images and colours in your browser.

2.1 Files compared

#	Location	File	Last Modified
1	PSGW_2.3_19.zip\PSGW_2.3_19\docs	EPIP_Test_Evaluation_(PSGW_2.3_19).doc	Tue Oct 17 17:04:50 2017 UTC
2	PSGW_2.3_19.zip\PSGW_2.3_19\docs	EPIP_Test_Evaluation_(PSGW_2.3_19).doc	Tue Oct 17 17:10:10 2017 UTC

2.2 Comparison summary

Description	Between Files 1 and 2
Description	Text Blocks	Lines
Unchanged	4	560
Changed	3	6
Inserted	0	0
Removed	0	0

2.3 Comparison options

Whitespace
Character case	Differences in character case are significant
Line endings	Differences in line endings (`CR` and `LF` characters) are ignored
CR/LF characters	Not shown in the comparison detail

2.4 Active regular expressions

No regular expressions were active.

2.5 Comparison detail

	1	Test Evalu ationExist ing Produc t Intake P rogram (EP IP)
	2	Patch PSGW 2.319
	3	Test Evalu ation
	4
	5	Department of Vetera ns Affairs
	6	August 201 7
	7	Version 1. 0
	8	Revision H istory
	9	Note: The revision h istory cyc le begins once chang es or enha ncements a re request ed after t he Communi cations Pl an has bee n baseline d.
	10	DateVersio nDescripti onAuthor08 /17/20171. 0Initial d ocument.EP IP Project TeamArtif act Ration ale
	11	The test e valuation document i s the prim ary output of the te st and eva luation pr ocess, an integral p art of the systems e ngineering process, which iden tifies lev els of per formance a nd assists the devel oper in co rrecting d eficiencie s.
	12	Table of C ontents
	13	11.
	14	Test Evalu ation Intr oduction
	15
	16
	17	11.1.
	18	Test Evalu ation Scop e
	19
	20
	21	11.2.
	22	Test Archi tecture
	23
	24
	25	11.3.
	26	Test Envir onment/Con figuration
	27
	28
	29	21.4.
	30	Installati on Process
	31
	32
	33	22.
	34	Test Data
	35
	36
	37	23.
	38	Issues
	39
	40
	41	24.
	42	Test Execu tion Log
	43
	44
	45	35.
	46	Test Defec t Log
	47
	48
	49	36.
	50	Test Resul ts Summary
	51
	52
	53	36.1.
	54	Defect Sev erity and Priority L evels
	55
	56
	57	46.2.
	58	Total Defe cts by Sev erity Leve l
	59
	60
	61	46.3.
	62	Breakdown of Test Re sults
	63
	64
	65	46.4.
	66	Performanc e Testing
	67
	68
	69	47.
	70	Test Cover age
	71
	72
	73	47.1.
	74	Requiremen ts Covered
	75
	76
	77	57.2.
	78	Section 50 8 Complian ce Coverag e
	79
	80
	81	58.
	82	Suggested Actions
	83
	84
	85	59.
	86	Defect Sev erity and Priority D efinitions
	87
	88
	89	59.1.
	90	Defect Sev erity Leve l
	91
	92
	93	59.1.1.
	94	Severity L evel 1 – C ritical
	95
	96
	97	69.1.2.
	98	Severity L evel 2 - H igh
	99
	100
	101	69.1.3.
	102	Severity L evel 3 - M edium
	103
	104
	105	69.1.4.
	106	Severity L evel 4 - L ow
	107
	108
	109	79.2.
	110	Priority C lassificat ions
	111
	112
	113	79.2.1.
	114	Priority 1 - Resolve Immediate ly
	115
	116
	117	79.2.2.
	118	Priority 2 - Give Hi gh Attenti on
	119
	120
	121	79.2.3.
	122	Priority 3 - Normal Queue
	123
	124
	125	79.2.4.
	126	Priority 4 - Low Pri ority
	127
	128
	129	710.
	130	Optional T ables, Cha rts, and G raphs
	131
	132
	133	811.
	134	Document A pproval Si gnatures
	135
	136
	137	9Appendix A - Test E xecution L og
	138
	139
	140	10Appendix B – Defec t Log
	141
	142
	143
	144
	145	Test Evalu ation Intr oduction
	146	The purpos e of this Test Evalu ation is t o:
	147	Identify t he testing approach used.
	148	Present a summary an alysis of the key te st results from the remediatio n of this intake for review an d assessme nt by desi gnated sta keholders.
	149	Provide a general st atement of the quali ty of the system und er test.
	150	Make recom mendations for futur e testing efforts.
	151	Test Evalu ation Scop e
	152	The scope of this Te st Evaluat ion is to verify the functiona lity of th e Patch PS GW2.319 code modif ication, a s determin ed by Func tional, Co mponent In tegration/ System, an d Regressi on testing . Testing activities followed the specif ications o utlined in the follo wing Maste r Test Pla n: PSGW2. 319 Maste r Test Pla n (include d in Appen dix A).
	153	Test Archi tecture
	154	Following are the EP IP test ac counts use d by the L eidos Deve lopment an d SQA Test ing teams to test PS GW2.319.
	155	Developmen t Test Acc ounts
	156	(For Unit Testing)SQ A Test Acc ounts
	157	(For Funct ional, Reg ression, a nd Compone nt Integra tion and S ystem Test ing)VistAS 1 ( altern ate name: D1S1)VistA G1 (altern ate name: D1G1) Vist AS2 (alter nate name: D1S2) –
	158	for CPRS G UI testing onlyVistA G2 (altern ate name: D1G2) –
	159	for CPRS G UI testing onlyTest Environmen t/Configur ation
	160	The EPIP t est accoun ts are mai ntained by the EPIP System Adm inistrator , who inst alls all V A-released patches a s soon as they are n ationally released. All EPIP t est accoun ts are clo ned from e xisting VA Enterpris e Testing Services ( ETS) test accounts. The Comput erized Pat ient Recor d System ( CPRS) Grap hical User Interface (GUI) exe cutable is configure d for each VistA ins tance util izing a un ique Inter net Protoc ol (IP) ad dress to c onnect to the VistA applicatio ns. Any up dates to t he CPRS GU I executab le are han dled by th e EPIP Sys tem Admini strator.
	161	All EPIP T est Engine ers and De velopers w ho have th e proper c redentials can acces s the test accounts. The VA Au stin Infor mation Tec hnology Ce nter (AITC ) support team reset s password s and sets up new ac cess crede ntials on an as-need ed basis.
	162	Installati on Process
	163	As soon as the remed iation pro cess is co mplete and the patch is availa ble for te sting, a K IDS build is created in the De velopment account an d then sen t to FORUM for final packaging . The patc h is then submitted to the VA SQA Lead’s Mailman a ccount for installat ion.
	164	An EPIP De veloper or Test Engi neer utili zes the KI DS Install ation proc ess to ext ract the b uild from the patch and instal l the buil d into a t est accoun t. The ind ividual wh o installs the patch verifies the routin e checksum s and also checks fo r errors d uring the installati on process . If the p atch is su ccessfully installed without a ny errors, then the EPIP Test team proce eds with F unctional, Regressio n, and Com ponent Int egration a nd System testing. I f defects are found, then the Developmen t team wor ks to find a resolut ion and cr eates new versions o f the patc h until al l defects are resolv ed.
	165	Test Data
	166	The SQA Te sting team utilizes the test d ata in the designate d test acc ounts (D1G 1, D1G2).
	167	The test d ata is enc rypted fol lowing the standards set forth by the VA Office of Informati on & Techn ology (OI& T). All Pe rsonally I dentifiabl e Informat ion (PII) and Protec ted Health Informati on (PHI) i s scrubbed and is no t availabl e to the T est Engine ers.
	168	All testin g is execu ted using encrypted test patie nts availa ble from a ny of the EPIP test accounts. Examples o f encrypte d test pat ients:
	169	AAAHURMMX, XPHY
	170	BADHB, HAA DXS
	171	FDHUX, YHI J
	172	All tests were execu ted manual ly by EPIP Test Engi neers.
	173	Issues
	174	The follow ing issues were enco untered du ring testi ng of PSGW 2.319.
	175	TitleIssue Descripti onTypeSeve rityN/AN/A N/AN/ATest Execution Log
	176	The Test E xecution L og records the execu tion of te st scripts and docum ents the t est result s for each test scri pt.
	177	The SQA Te sting team utilizes the Ration al Quality Managemen t (QM) too l for all testing ac tivities. All test d ocuments a re stored in the EPI P reposito ry, includ ing the Ma ster Test Plan, Test Suites, T est Cases, and Test Scripts. T est execut ion is per formed, an d test res ults recor ded, in Ra tional QM. The Test Engineer a dds the te st results to the Te st Executi on records to indica te whether testing a chieved Pa ss or Fail status.
	178	The Test E xecution r ecords for PSGW2.3 19 are inc luded in t he EPIP Pa tch PSGW2 .319 Mast er Test Pl an. The Ma ster Test Plan is av ailable in Appendix A.
	179	Test Defec t Log
	180	The Test D efect Log is a tool for record ing, analy zing, trac king, and documentin g the clos ure of def ects. It s pecifies t he screen, field, be havior or result tha t occurred , and the IEEE-defin ed Severit y Level. I t includes enough in formation for the de veloper to find and re-create the defect . The Defe ct Log is available in Appendi x B.
	181	Test Resul ts Summary
	182	SQA testin g for this intake st arted in t he Dev1 Go ld1 test e nvironment on July 2 0, 2017 an d ended on August 16 , 2017.
	183	Test versi on 1 was i nstalled i n the Dev1 Gold1 tes t environm ent after Unit testi ng in Dev1 Silver1 w as complet ed. Upon c ompletion of Integra tion testi ng (Compon ent Integr ation and System Tes ting, Func tional Tes ting, and Regression Testing), one (1) d efect was found and reported.
	184	Test versi on 2 was i nstalled i n the Dev1 Gold1 tes t environm ent on Jul y 21, 2017 after Uni t testing in Dev1 Si lver1 was completed. This vers ion correc ted an iss ue with th e on/off s witch para meter bein g set to n ull and tw o typos in the displ ay. Upon c ompletion of Functio nal Testin g, one (1) defect wa s found an d reported .
	185	Test versi on 3 was i nstalled i n the Dev1 Gold1 tes t environm ent on Jul y 25, 2017 after Uni t testing in Dev1 Si lver1 was completed. This vers ion correc ted the is sue that t he changes made in t he previou s test ver sion were not propag ated to FO RUM. Upon completion of Functi onal Testi ng, one (1 ) defect w as found a nd reporte d.
	186	Test versi on 4 was i nstalled i n the Dev1 Gold1 tes t environm ent on Aug ust 7, 201 7 after Un it testing in Dev1 S ilver1 was completed . This ver sion corre cted two i ssues rela ted to nur ses’ order ing items. Upon comp letion of Integratio n testing (Component Integrati on and Sys tem Testin g, Functio nal Testin g, and Reg ression Te sting), ze ro (0) def ects were found and reported. During an internal d emonstrati on, a conf lict with a patch in stalled in to the tes t environm ent after testing wa s complete d was disc overed. On e (1) defe ct was rep orted.
	187	Test versi on 5 was i nstalled i n the Dev1 Gold1 tes t environm ent on Aug ust 15, 20 17 after U nit testin g in Dev1 Silver1 wa s complete d. This ve rsion corr ected an i ssue with the Stock Level not displaying in the On Demand Re port by Da te/AOU. Up on complet ion of Int egration t esting (Co mponent In tegration and System Testing, Functional Testing, and Regres sion Testi ng), zero (0) defect s were fou nd and rep orted.
	188	Defect Sev erity and Priority L evels
	189	A defect i s defined as a flaw in a compo nent or sy stem that can cause the compon ent or sys tem to fai l to perfo rm its req uired func tion, e.g. , an incor rect state ment or da ta definit ion. A def ect, if en countered during exe cution, ma y cause a failure of the compo nent or sy stem.
	190	Defects ar e categori zed accord ing to sev erity and priority l evels. The test anal yst assign s the seve rity, whil e the deve lopment ma nager assi gns the pr iority for repair. F or more in formation, see Defec t Severity and Prior ity Defini tion in th is Test Ev aluation.
	191	Total Defe cts by Sev erity Leve l
	192	The Defect Log in Ap pendix B d isplays th e defects encountere d while te sting this patch, an d the seve rity level of each.
	193	Breakdown of Test Re sults
	194	Testing wa s complete d on Augus t 16, 2017 . All test results w ere record ed in Rati onal QM. D etailed re sults are available in the EPI P Patch PS GW2.319 Master Tes t Plan (se e Appendix A).
	195	Performanc e Testing
	196	Performanc e testing was not co nducted.
	197	Test Cover age
	198	The EPIP P atch PSGW* 2.3*19 Mas ter Test P lan contai ns details on test c overage (s ee Appendi x A).
	199	Requiremen ts Covered
	200	The requir ements for PSGW2.3 19 are sto red in the Rational Requiremen ts Managem ent (RM) a pplication . The test cases sto red in Rat ional Qual ity Manage ment (QM) are used t o validate that the requiremen ts have be en address ed, provid ing full t raceabilit y. The use r stories stored in Rational C onfigurati on Managem ent (CM) a re linked to the req uirements in RM and test cases in QM.
	201	The follow ing links provide ac cess to th e various ARWS repos itories in the Ratio nal toolki t. If link translati on issues prevent di rect acces s, copy an d paste th e URLs int o your bro wser.
	202	ARWS (RM) – Go to Ar tifacts, t hen Browse Artifacts . Locate t he EPIP fo lder on th e left sid e of the p age and ex pand it to display p atch folde rs. Each p atch folde r contains the requi rements fo r the patc h number s hown in th e folder n ame.
	203	https://cl m.rational .oit. DNS /rm/web#ac tion=com.i bm.rdm.web .pages.sho wFoundatio nProjectDa shboard&pr ojectURI=h ttps%3A%2F %2Fclm.rat ional.oit. DNS %2Frm%2Fpr ocess%2Fpr oject-area s%2F_cOYOU XtkEea6E-3 s3yro6A&vv c.configur ation=http s%3A%2F%2F clm.ration al.oit. DNS %2Frm%2Fcm %2Fstream% 2F_c9605nt kEea6E-3s3 yro6A
	204	ARWS (QM) – Go to Pl anning, th en Browse Test Plans , and then search fo r the Mast er Test Pl an you nee d. The Mas ter Test P lan and te st cases a re linked to require ments.
	205	https://cl m.rational .oit. DNS /qm/web/co nsole/ARWS %20(QM)#ac tion=com.i bm.rqm.pla nning.home .actionDis patcher&su bAction=vi ewUserHome
	206	ARWS (CM) – Go to Pl ans, then All Plans, and then search for the Sprin t Plan you need. The user stor ies in eac h Plan are linked to requireme nts and te st cases.
	207	https://cl m.rational .oit. DNS /ccm/web/p rojects/AR WS%20(CM)# action=com .ibm.team. dashboard. viewDashbo ard
	208	Section 50 8 Complian ce Coverag e
	209	Section 50 8 test res ults will be reporte d to VA in the follo wing docum ents:
	210	EPIP_VASec tion508_Co mpliance_T est_Result s_(PSGW2. 319)
	211	EPIP_VASec tion508_Co nformance_ Validation _Statement _( PSGW2. 319)
	212	EPIP_VASec tion508_Se lf-Certifi cation_Doc ument_(PSG W2.319)
	213	EPIP_VASec tion508_Ve rifiable_O bjective_E vidence_( PSGW2.31 9)
	214	Suggested Actions
	215	Leidos rec ommends mo ving this patch to I OC testing .
	216	Defect Sev erity and Priority D efinitions
	217	The classi fication o f defects within a s ystem exam ines both the severi ty and pri ority of t he defect.
	218	Severity i s a measur e of how g reat the i mpact is o n the user ’s ability to comple te the doc umented ac tions with in the sys tem.
	219	Priority d etermines the speed with which a given d efect must be repair ed.
	220	Defect cla ssificatio n may be d etermined either bec ause testi ng is dela yed by a f ailure in the system or becaus e a cumber some worka round prev ents a use r from com pleting th e assigned tasks. Bo th severit y and prio rity measu res must b e recorded when sche duling def ect resolu tion tasks .
	221	Defect Sev erity Leve l
	222	The follow ing subsec tions iden tify the d efect seve rity level s.
	223	Severity L evel 1 – C ritical
	224	Institute of Electri cal and El ectronics Engineers (IEEE) def inition: T he defect results in the failu re of the complete s oftware sy stem, of a subsystem , or of a software u nit (progr am or modu le) within the syste m.
	225	Any defect that comp romises pa tient safe ty or syst em securit y. Example s of syste m security defects i nclude bre ach of con fidentiali ty require ments of t he Privacy Act, the Health Ins urance Por tability a nd Account ability Ac t (HIPAA), or Federa l Tax Info rmation gu idelines.
	226	Loss of sy stem funct ionality c ritical to user oper ations wit h no suita ble workar ound, i.e. , there is no way to achieve t he expecte d results using the applicatio n.
	227	System cra sh or hang that prev ents furth er testing or operat ion of the complete applicatio n or a sec tion of th e applicat ion.
	228	Any defect that caus es corrupt ion of dat a from a r esult of t he system (as oppose d to user error).
	229	Any defect in which inappropri ate transm issions ar e consiste ntly gener ated or ap propriate transmissi ons of HL7 messages fail to be generated .
	230	Loss of fu nctionalit y resultin g in erron eous eligi bility/enr ollment de terminatio ns or comm unications not being sent.
	231	Severity L evel 2 - H igh
	232	IEEE defin ition: The defect re sults in t he failure of the co mplete sof tware syst em, of a s ubsystem, or of a so ftware uni t (program or module ) within t he system. There is no way to make the f ailed comp onent(s) f unction. H owever, th ere are ac ceptable p rocessing alternativ es which w ill yield the desire d result.
	233	A major de fect in th e function ality that does not result in corruption of data.
	234	A major de fect in th e function ality resu lting in a failure o f all or p art of the applicati on, where:
	235	The expect ed results can tempo rarily be achieved b y alternat e means. T he custome r indicate s the work around is acceptabl e for the short term .
	236	Any defect that does not confo rm to Sect ion 508 st andards.
	237	Any defect that resu lts in ina ccurate or missing r equirement s.
	238	Any defect that resu lts in inv alid authe ntication or authent ication of an invali d end user .
	239	Severity L evel 3 - M edium
	240	IEEE defin ition: The defect do es not res ult in a f ailure, bu t causes t he system to produce incorrect , incomple te, or inc onsistent results, o r the defe ct impairs the syste ms usabili ty.
	241	Minor func tionality is not wor king as in tended and a workaro und exists but is no t suitable for long term use
	242	The inabil ity of a v alid user to access the system consisten t with gra nted privi leges
	243	Typographi cal or gra mmatical e rrors in t he applica tion, incl uding inst allation g uides, use r guides, training m anuals, an d design d ocuments
	244	Any defect producing cryptic, incorrect, or inappr opriate er ror messag es
	245	Any defect that resu lts from t he use of non-standa rd data te rminology in the app lication o r document ation, as defined by the Depar tment of V eterans Af fairs
	246	Cosmetic i ssues that are impor tant to th e integrit y of the p roduct, bu t do not r esult in d ata entry and or dat a quality problems.
	247	Severity L evel 4 - L ow
	248	IEEE defin ition: The defect do es not cau se a failu re, does n ot impair usability, and the d esired pro cessing re sults are easily obt ained by w orking aro und the de fect.
	249	Minor loss of, or de fect in th e function ality wher e a long t erm use ex ists
	250	Low-level cosmetic i ssues.
	251	Priority C lassificat ions
	252	The follow ing subsec tions iden tify the a ppropriate actions f or defects at each p riority le vel, per d efinitions of IEEE.
	253	Priority 1 - Resolve Immediate ly
	254	Further de velopment and/or tes ting canno t occur un til the de fect has b een repair ed. The sy stem canno t be used until the repair has been affe cted.
	255	Priority 2 - Give Hi gh Attenti on
	256	The defect must be r esolved as soon as p ossible be cause it i s impairin g developm ent and/or testing a ctivities. System us e will be severely a ffected un til the de fect is fi xed.
	257	Priority 3 - Normal Queue
	258	The defect should be resolved in the nor mal course of develo pment acti vities. It can wait until a ne w build or version i s created.
	259	Priority 4 - Low Pri ority
	260	The defect is an irr itant that should be repaired, but can b e repaired after mor e serious defects ha ve been fi xed.
	261	Optional T ables, Cha rts, and G raphs
	262	None.
	263	Document A pproval Si gnatures
	264	Signed: __ __________ __________ __________ __________ __________ __________ _________
	265	Program/Pr oject Mana ger
	266	Date
	267	Signed: __ __________ __________ __________ __________ __________ __________ _________
	268	Business S ponsor Rep resentativ e
	269	Date
	270	Signed: __ __________ __________ __________ __________ __________ __________ _________
	271	Test Lead
	272	Date
	273	Appendix A - Test Ex ecution Lo g
	274	The Test E xecution R ecords for PSGW2.3 19 are inc luded in t he EPIP Pa tch PSGW2 .319 Mast er Test Pl an.
	275
	276	Appendix B – Defect Log
	277	Four (4) d efects wer e found du ring testi ng of PSGW 2.319. A ll defects have been resolved.
	278	Defect IDA ffected Sc reenAffect ed FieldOb served Beh aviorSever ityDescrip tionN/AN/A N/AN/AN/AN o defects were found during Un it Testing of versio n 1.N/AN/A N/AN/AN/AN o defects were found during Co mponent In tegration/ Systems Te sting of v ersion 1.N /AN/AN/AN/ AN/ANo def ects were found duri ng Regress ion testin g of versi on 1.56337 7Edit Para meter Valu esWS_LVL_O N Paramete rIssue 1) The parame ter WS_LVL _ON should be OFF by default w hen instal led; as of now it is NULL. The options Y ES or NO a re also no t being di splayed.
	279	Issue 2) T ypo ? at W ARD STOCK LEVEL:
	280	Issue 3) T ypo ?? at WARD STOCK LEVEL: Pl ease fix t he text he re.LowOne defect was found dur ing Functi onal Testi ng of vers ion 1.N/AN /AN/AN/AN/ ANo defect s were fou nd during Unit Testi ng of vers ion 2.5624 92N/AN/ATh e changes that were supposed t o be refle cted in Ve rsion 2 we re not pro pagated to FORUM.Med iumOne def ect was fo und during Functiona l Testing of version 2.N/AN/AN /AN/AN/ANo defects w ere found during Uni t Testing of version 3.566638E nter/Edit Nurses' On -Demand Re quest
	281	ON-DEMAND QUANTITYIs sue 1) Nur ses’ order s with NUL L for the requested quantity a re printed in the Ph armacists On Demand report.
	282	Issue 2) W hile nurse is enteri ng orders, if for so me reason the nurse abruptly q uits after a few ite ms using ^ , the comp leted item s are not being sent to the ph armacist’s printer. The print code is no t being re ached sinc e the exit was abrup t.MediumOn e defect w as found d uring Func tional Tes ting of ve rsion 3.N/ AN/AN/AN/A N/ANo defe cts were f ound durin g Unit Tes ting of ve rsion 4.N/ AN/AN/AN/A N/ANo defe cts were f ound durin g Componen t Integrat ion/System s Testing of version 4.N/AN/AN /AN/AN/ANo defects w ere found during Reg ression te sting of v ersion 4.N /AN/AN/AN/ AN/ANo def ects were found duri ng Functio nal Testin g of versi on 4.Defec t ID 57316 9 was logg ed for an issue that was not f ound durin g initial testing, b ut discove red after another pa tch (PSGW* 2.3*18) wa s installe d in the t est enviro nment. Sto ck Level w as not bei ng display ed in the Print an O n Demand R eport by D ate/AOU du e to the i nstallatio n of that patch.N/AN /AN/AN/AN/ ANo defect s were fou nd during Unit Testi ng of vers ion 5.N/AN /AN/AN/AN/ ANo defect s were fou nd during Component Integratio n/Systems Testing of version 5 .N/AN/AN/A N/AN/ANo d efects wer e found du ring Regre ssion test ing of ver sion 5.N/A N/AN/AN/A N/ANo defe cts were f ound durin g Function al Testing of versio n 5.
	283	_156974703 0.pdf