Produced by Araxis Merge on 2/19/2019 12:20:55 PM Central Standard Time. See www.araxis.com for information about Merge. This report uses XHTML and CSS2, and is best viewed with a modern standards-compliant browser. For optimum results when printing this report, use landscape orientation and enable printing of background images and colours in your browser.
# | Location | File | Last Modified |
---|---|---|---|
1 | C:\AraxisMergeCompare\Pri_un\EPIP combined\GMTS_2.7_124 | EPIP_Test_Evaluation_(GMTS_2.7_124).doc | Tue Feb 12 17:11:15 2019 UTC |
2 | C:\AraxisMergeCompare\Pri_re\EPIP combined\GMTS_2.7_124 | EPIP_Test_Evaluation_(GMTS_2.7_124).doc | Tue Feb 12 18:23:56 2019 UTC |
Description | Between Files 1 and 2 |
|
---|---|---|
Text Blocks | Lines | |
Unchanged | 4 | 548 |
Changed | 3 | 6 |
Inserted | 0 | 0 |
Removed | 0 | 0 |
Whitespace | |
---|---|
Character case | Differences in character case are significant |
Line endings | Differences in line endings (CR and LF characters) are ignored |
CR/LF characters | Not shown in the comparison detail |
No regular expressions were active.
1 | EPIP_Test_ Evaluation Existing P roduct Int ake Progra m (EPIP) | |
2 | Patch GMTS *2.7*124 | |
3 | Test Evalu ation | |
4 | ||
5 | Department of Vetera ns Affairs | |
6 | January 20 19 | |
7 | Version 1. 0 | |
8 | Revision H istory | |
9 | Note: The revision h istory cyc le begins once chang es or enha ncements a re request ed after t he Communi cations Pl an has bee n baseline d. | |
10 | DateVersio nDescripti onAuthor01 /24/20191. 0Initial d ocument.EP IP Project TeamArtif act Ration ale | |
11 | The test e valuation document i s the prim ary output of the te st and eva luation pr ocess, an integral p art of the systems e ngineering process, which iden tifies lev els of per formance a nd assists the devel oper in co rrecting d eficiencie s. | |
12 | Table of C ontents | |
13 | 11. | |
14 | Test Evalu ation Intr oduction | |
15 | ||
16 | ||
17 | 11.1. | |
18 | Test Evalu ation Scop e | |
19 | ||
20 | ||
21 | 11.2. | |
22 | Test Archi tecture | |
23 | ||
24 | ||
25 | 11.3. | |
26 | Test Envir onment/Con figuration | |
27 | ||
28 | ||
29 | 21.4. | |
30 | Installati on Process | |
31 | ||
32 | ||
33 | 22. | |
34 | Test Data | |
35 | ||
36 | ||
37 | 23. | |
38 | Issues | |
39 | ||
40 | ||
41 | 24. | |
42 | Test Execu tion Log | |
43 | ||
44 | ||
45 | 35. | |
46 | Test Defec t Log | |
47 | ||
48 | ||
49 | 36. | |
50 | Test Resul ts Summary | |
51 | ||
52 | ||
53 | 36.1. | |
54 | Defect Sev erity and Priority L evels | |
55 | ||
56 | ||
57 | 36.2. | |
58 | Total Defe cts by Sev erity Leve l | |
59 | ||
60 | ||
61 | 36.3. | |
62 | Breakdown of Test Re sults | |
63 | ||
64 | ||
65 | 36.4. | |
66 | Performanc e Testing | |
67 | ||
68 | ||
69 | 37. | |
70 | Test Cover age | |
71 | ||
72 | ||
73 | 37.1. | |
74 | Requiremen ts Covered | |
75 | ||
76 | ||
77 | 47.2. | |
78 | Section 50 8 Complian ce Coverag e | |
79 | ||
80 | ||
81 | 48. | |
82 | Suggested Actions | |
83 | ||
84 | ||
85 | 49. | |
86 | Defect Sev erity and Priority D efinitions | |
87 | ||
88 | ||
89 | 59.1. | |
90 | Defect Sev erity Leve l | |
91 | ||
92 | ||
93 | 59.1.1. | |
94 | Severity L evel 1 – C ritical | |
95 | ||
96 | ||
97 | 59.1.2. | |
98 | Severity L evel 2 - H igh | |
99 | ||
100 | ||
101 | 69.1.3. | |
102 | Severity L evel 3 - M edium | |
103 | ||
104 | ||
105 | 69.1.4. | |
106 | Severity L evel 4 - L ow | |
107 | ||
108 | ||
109 | 69.2. | |
110 | Priority C lassificat ions | |
111 | ||
112 | ||
113 | 69.2.1. | |
114 | Priority 1 - Resolve Immediate ly | |
115 | ||
116 | ||
117 | 69.2.2. | |
118 | Priority 2 - Give Hi gh Attenti on | |
119 | ||
120 | ||
121 | 69.2.3. | |
122 | Priority 3 - Normal Queue | |
123 | ||
124 | ||
125 | 69.2.4. | |
126 | Priority 4 - Low Pri ority | |
127 | ||
128 | ||
129 | 710. | |
130 | Optional T ables, Cha rts, and G raphs | |
131 | ||
132 | ||
133 | 811. | |
134 | Document A pproval Si gnatures | |
135 | ||
136 | ||
137 | 9Appendix A - Test E xecution L og | |
138 | ||
139 | ||
140 | 10Appendix B – Defec t Log | |
141 | ||
142 | ||
143 | ||
144 | ||
145 | Test Evalu ation Intr oduction | |
146 | The purpos e of this Test Evalu ation is t o: | |
147 | Identify t he testing approach used. | |
148 | Present a summary an alysis of the key te st results from the remediatio n of this intake for review an d assessme nt by desi gnated sta keholders. | |
149 | Provide a general st atement of the quali ty of the system und er test. | |
150 | Make recom mendations for futur e testing efforts. | |
151 | Test Evalu ation Scop e | |
152 | The scope of this Te st Evaluat ion is to verify the functiona lity of th e Patch GM TS*2.7*124 code modi fication, as determi ned by Fun ctional, C omponent I ntegration /System, a nd Regress ion testin g. Testing activitie s followed the speci fications outlined i n the foll owing Mast er Test Pl an: GMTS*2 .7*124 Mas ter Test P lans (incl uded in Ap pendix A). | |
153 | Test Archi tecture | |
154 | Following are the EP IP test ac counts use d by the L eidos Deve lopment an d SQA Test ing teams to test GM TS*2.7*124 . | |
155 | Developmen t Test Acc ounts | |
156 | (For Unit Testing)SQ A Test Acc ounts | |
157 | (For Funct ional, Reg ression, a nd Compone nt Integra tion and S ystem Test ing)VistAS 1 (alterna te name: D 1S1)VistAG 1 (alterna te name: D 1G1) VistA S2 (altern ate name: D1S2) – | |
158 | for CPRS G UI testing onlyVistA G2 (altern ate name: D1G2) – | |
159 | for CPRS G UI testing onlyTest Environmen t/Configur ation | |
160 | The EPIP t est accoun ts are mai ntained by the EPIP System Adm inistrator , who inst alls all V A-released patches a s soon as they are n ationally released. All EPIP t est accoun ts are clo ned from e xisting VA Enterpris e Testing Services ( ETS) test accounts. The Comput erized Pat ient Recor d System ( CPRS) Grap hical User Interface (GUI) exe cutable is configure d for each VistA ins tance util izing a un ique Inter net Protoc ol (IP) ad dress to c onnect to the VistA applicatio ns. Any up dates to t he CPRS GU I executab le are han dled by th e EPIP Sys tem Admini strator. | |
161 | All EPIP T est Engine ers and De velopers w ho have th e proper c redentials can acces s the test accounts. The VA Au stin Infor mation Tec hnology Ce nter (AITC ) support team reset s password s and sets up new ac cess crede ntials on an as-need ed basis. | |
162 | Installati on Process | |
163 | As soon as the remed iation pro cess is co mplete and the patch is availa ble for te sting, a K IDS build is created in the De velopment account an d then sen t to FORUM for final packaging . The patc h is then submitted to the VA SQA Lead’s Mailman a ccount for installat ion. | |
164 | An EPIP De veloper or Test Engi neer utili zes the KI DS Install ation proc ess to ext ract the b uild from the patch and instal l the buil d into a t est accoun t. The ind ividual wh o installs the patch verifies the routin e checksum s and also checks fo r errors d uring the installati on process . If the p atch is su ccessfully installed without a ny errors, then the EPIP Test team proce eds with F unctional, Regressio n, and Com ponent Int egration a nd System testing. I f defects are found, then the Developmen t team wor ks to find a resolut ion and cr eates new versions o f the patc h until al l defects are resolv ed. | |
165 | Test Data | |
166 | The SQA Te sting team utilizes the test d ata in the designate d test acc ounts (D1G 1, D1G2). | |
167 | The test d ata is enc rypted fol lowing the standards set forth by the VA Office of Informati on & Techn ology (OIT ). All Per sonally Id entifiable Informati on (PII) a nd Protect ed Health Informatio n (PHI) is scrubbed and is not available to the Te st Enginee rs. | |
168 | All testin g is execu ted using encrypted test patie nts availa ble from a ny of the EPIP test accounts. Examples o f encrypte d test pat ients: | |
169 | AAAHURMMX, XPHY | |
170 | BADHB, HAA DXS | |
171 | FDHUX, YHI J | |
172 | All tests were execu ted manual ly by EPIP Test Engi neers. | |
173 | Issues | |
174 | No issues were encou ntered dur ing testin g of GMTS* 2.7*124. | |
175 | TitleIssue Descripti onTypeSeve rityN/AN/A N/AN/ATest Execution Log | |
176 | The Test E xecution L og records the execu tion of te st scripts and docum ents the t est result s for each test scri pt. | |
177 | The SQA Te sting team utilizes the Ration al Quality Managemen t (QM) too l for all testing ac tivities. All test d ocuments a re stored in the EPI P reposito ry, includ ing the Ma ster Test Plan, Test Suites, T est Cases, and Test Scripts. T est execut ion is per formed, an d test res ults recor ded, in Ra tional QM. The Test Engineer a dds the te st results to the Te st Executi on records to indica te whether testing a chieved Pa ss or Fail status. | |
178 | The Test E xecution r ecords for GMTS*2.7* 124 are in cluded in the EPIP P atch GMTS* 2.7*124 Ma ster Test Plan. The Master Tes t Plan is available in Appendi x A. | |
179 | Test Defec t Log | |
180 | The Test D efect Log is a tool for record ing, analy zing, trac king, and documentin g the clos ure of def ects. It s pecifies t he screen, field, be havior or result tha t occurred , and the IEEE-defin ed Severit y Level. I t includes enough in formation for the de veloper to find and re-create the defect . The Defe ct Log is available in Appendi x B. | |
181 | Test Resul ts Summary | |
182 | SQA testin g for this patch sta rted on No vember 15, 2018 and ended on D ecember 21 , 2018. | |
183 | Test versi on 1 was i nstalled i n the Dev1 Gold1 sys tem after Unit testi ng in Dev1 Silver1 w as complet ed. Upon c ompletion of Integra tion testi ng (Compon ent Integr ation and System Tes ting, Func tional Tes ting, and Regression Testing), zero (0) defects we re found a nd reporte d. | |
184 | Defect Sev erity and Priority L evels | |
185 | A defect i s defined as a flaw in a compo nent or sy stem that can cause the compon ent or sys tem to fai l to perfo rm its req uired func tion, e.g. , an incor rect state ment or da ta definit ion. A def ect, if en countered during exe cution, ma y cause a failure of the compo nent or sy stem. | |
186 | Defects ar e categori zed accord ing to sev erity and priority l evels. The test anal yst assign s the seve rity, whil e the deve lopment ma nager assi gns the pr iority for repair. F or more in formation, see Defec t Severity and Prior ity Defini tion in th is Test Ev aluation. | |
187 | Total Defe cts by Sev erity Leve l | |
188 | The Defect Log in Ap pendix B d isplays th e defects encountere d while te sting this patch, an d the seve rity level of each. | |
189 | Breakdown of Test Re sults | |
190 | Testing wa s complete d on Decem ber 21, 20 18. All te st results were reco rded in Ra tional QM. Detailed results ar e availabl e in the E PIP Patch GMTS*2.7*1 24 Master Test Plan (see Appen dix A). | |
191 | Performanc e Testing | |
192 | Performanc e testing was not co nducted. | |
193 | Test Cover age | |
194 | The EPIP P atch GMTS* 2.7*124 Ma ster Test Plan conta ins detail s on test coverage ( see Append ix A). | |
195 | Requiremen ts Covered | |
196 | The requir ements for GMTS*2.7* 124 are st ored in th e Rational Requireme nts Manage ment (RM) applicatio n. The tes t cases st ored in Ra tional Qua lity Manag ement (QM) are used to validat e that the requireme nts have b een addres sed, provi ding full traceabili ty. The us er stories stored in Rational Configurat ion Manage ment (CM) are linked to the re quirements in RM and test case s in QM. | |
197 | The follow ing links provide ac cess to th e various Health Sum mary (HS) repositori es in the Rational t oolkit. If link tran slation is sues preve nt direct access, co py and pas te the URL s into you r browser. | |
198 | HS (RM) – Go to Arti facts. Loc ate the EP IP folder at the lef t side of the page a nd expand it to disp lay patch folders. E ach patch folder con tains the requiremen ts for the patch num ber shown in the fol der name. | |
199 | https:// URL /rm/web#ac tion=com.i bm.rdm.web .pages.sho wFoundatio nProjectDa shboard&co mponentURI =https:// URL /rm/rm-pro jects/_PB7 A0B6uEeafn 8RFT4tzpQ/ components /_PWMAoB6u Eeafn8RFT4 tzpQ | |
200 | HS (QM) – Go to Plan ning, then Browse Te st Plans, and then s earch for the Master Test Plan you need. The Maste r Test Pla n and test cases are linked to requireme nts. | |
201 | https:// URL /qm/web/co nsole/HS%2 0(QM)#acti on=com.ibm .rqm.plann ing.home.a ctionDispa tcher&subA ction=view UserHome | |
202 | HS (CM) – Go to Plan s, then Al l Plans, a nd then se arch for t he Sprint Plan you n eed. The u ser storie s in each Plan are l inked to r equirement s and test cases. | |
203 | https:// URL /ccm/web/p rojects/HS %20(CM)#ac tion=com.i bm.team.da shboard.vi ewDashboar d | |
204 | Section 50 8 Complian ce Coverag e | |
205 | Section 50 8 test res ults will be reporte d to VA in the follo wing docum ents: | |
206 | EPIP_VASec tion508_Co mpliance_T est_Result s_(GMTS_2. 7_124) | |
207 | EPIP_VASec tion508_In take_Docum ent_(GMTS_ 2.7_124) | |
208 | EPIP_VASec tion508_Ve rifiable_O bjective_E vidence_(G MTS_2.7_12 4) | |
209 | EPIP_VASec tion508_11 94_21_Soft ware_Check list_(GMTS _2.7_124) | |
210 | EPIP_VASec tion508_11 94_31_Func tional_Che cklist_(GM TS_2.7_124 ) | |
211 | EPIP_VASec tion508_11 94_41_Docu mentation_ Checklist_ (GMTS_2.7_ 124) | |
212 | Suggested Actions | |
213 | Leidos rec ommends mo ving this patch to I OC testing . | |
214 | Defect Sev erity and Priority D efinitions | |
215 | The classi fication o f defects within a s ystem exam ines both the severi ty and pri ority of t he defect. | |
216 | Severity i s a measur e of how g reat the i mpact is o n the user ’s ability to comple te the doc umented ac tions with in the sys tem. | |
217 | Priority d etermines the speed with which a given d efect must be repair ed. | |
218 | Defect cla ssificatio n may be d etermined either bec ause testi ng is dela yed by a f ailure in the system or becaus e a cumber some worka round prev ents a use r from com pleting th e assigned tasks. Bo th severit y and prio rity measu res must b e recorded when sche duling def ect resolu tion tasks . | |
219 | Defect Sev erity Leve l | |
220 | The follow ing subsec tions iden tify the d efect seve rity level s. | |
221 | Severity L evel 1 – C ritical | |
222 | Institute of Electri cal and El ectronics Engineers (IEEE) def inition: T he defect results in the failu re of the complete s oftware sy stem, of a subsystem , or of a software u nit (progr am or modu le) within the syste m. | |
223 | Any defect that comp romises pa tient safe ty or syst em securit y. Example s of syste m security defects i nclude bre ach of con fidentiali ty require ments of t he Privacy Act, the Health Ins urance Por tability a nd Account ability Ac t (HIPAA), or Federa l Tax Info rmation gu idelines. | |
224 | Loss of sy stem funct ionality c ritical to user oper ations wit h no suita ble workar ound, i.e. , there is no way to achieve t he expecte d results using the applicatio n. | |
225 | System cra sh or hang that prev ents furth er testing or operat ion of the complete applicatio n or a sec tion of th e applicat ion. | |
226 | Any defect that caus es corrupt ion of dat a from a r esult of t he system (as oppose d to user error). | |
227 | Any defect in which inappropri ate transm issions ar e consiste ntly gener ated or ap propriate transmissi ons of HL7 messages fail to be generated . | |
228 | Loss of fu nctionalit y resultin g in erron eous eligi bility/enr ollment de terminatio ns or comm unications not being sent. | |
229 | Severity L evel 2 - H igh | |
230 | IEEE defin ition: The defect re sults in t he failure of the co mplete sof tware syst em, of a s ubsystem, or of a so ftware uni t (program or module ) within t he system. There is no way to make the f ailed comp onent(s) f unction. H owever, th ere are ac ceptable p rocessing alternativ es which w ill yield the desire d result. | |
231 | A major de fect in th e function ality that does not result in corruption of data. | |
232 | A major de fect in th e function ality resu lting in a failure o f all or p art of the applicati on, where: | |
233 | The expect ed results can tempo rarily be achieved b y alternat e means. T he custome r indicate s the work around is acceptabl e for the short term . | |
234 | Any defect that does not confo rm to Sect ion 508 st andards. | |
235 | Any defect that resu lts in ina ccurate or missing r equirement s. | |
236 | Any defect that resu lts in inv alid authe ntication or authent ication of an invali d end user . | |
237 | Severity L evel 3 - M edium | |
238 | IEEE defin ition: The defect do es not res ult in a f ailure, bu t causes t he system to produce incorrect , incomple te, or inc onsistent results, o r the defe ct impairs the syste ms usabili ty. | |
239 | Minor func tionality is not wor king as in tended and a workaro und exists but is no t suitable for long term use | |
240 | The inabil ity of a v alid user to access the system consisten t with gra nted privi leges | |
241 | Typographi cal or gra mmatical e rrors in t he applica tion, incl uding inst allation g uides, use r guides, training m anuals, an d design d ocuments | |
242 | Any defect producing cryptic, incorrect, or inappr opriate er ror messag es | |
243 | Any defect that resu lts from t he use of non-standa rd data te rminology in the app lication o r document ation, as defined by the Depar tment of V eterans Af fairs | |
244 | Cosmetic i ssues that are impor tant to th e integrit y of the p roduct, bu t do not r esult in d ata entry and or dat a quality problems. | |
245 | Severity L evel 4 - L ow | |
246 | IEEE defin ition: The defect do es not cau se a failu re, does n ot impair usability, and the d esired pro cessing re sults are easily obt ained by w orking aro und the de fect. | |
247 | Minor loss of, or de fect in th e function ality wher e a long t erm use ex ists | |
248 | Low-level cosmetic i ssues. | |
249 | Priority C lassificat ions | |
250 | The follow ing subsec tions iden tify the a ppropriate actions f or defects at each p riority le vel, per d efinitions of IEEE. | |
251 | Priority 1 - Resolve Immediate ly | |
252 | Further de velopment and/or tes ting canno t occur un til the de fect has b een repair ed. The sy stem canno t be used until the repair has been affe cted. | |
253 | Priority 2 - Give Hi gh Attenti on | |
254 | The defect must be r esolved as soon as p ossible be cause it i s impairin g developm ent and/or testing a ctivities. System us e will be severely a ffected un til the de fect is fi xed. | |
255 | Priority 3 - Normal Queue | |
256 | The defect should be resolved in the nor mal course of develo pment acti vities. It can wait until a ne w build or version i s created. | |
257 | Priority 4 - Low Pri ority | |
258 | The defect is an irr itant that should be repaired, but can b e repaired after mor e serious defects ha ve been fi xed. | |
259 | Optional T ables, Cha rts, and G raphs | |
260 | None. | |
261 | Document A pproval Si gnatures | |
262 | Signed: __ __________ __________ __________ __________ __________ __________ _________ | |
263 | Program/Pr oject Mana ger | |
264 | Date | |
265 | Signed: __ __________ __________ __________ __________ __________ __________ _________ | |
266 | Business S ponsor Rep resentativ e | |
267 | Date | |
268 | Signed: __ __________ __________ __________ __________ __________ __________ _________ | |
269 | Test Lead | |
270 | Date | |
271 | Appendix A - Test Ex ecution Lo g | |
272 | The Test E xecution R ecords for GMTS*2.7* 124 are in cluded in the EPIP P atch GMTS* 2.7*124 Ma ster Test Plan. | |
273 | ||
274 | Appendix B – Defect Log | |
275 | No defects were foun d during t esting of GMTS*2.7*1 24. | |
276 | Defect IDA ffected Sc reenAffect ed FieldOb served Beh aviorSever ityDescrip tionN/AN/A N/AN/AN/AN o defects were found during Un it Testing of versio n 1.0. N/A N/AN/AN/AN /ANo defec ts were fo und during Component Integrati on/System Testing of version 1 .0.N/AN/AN /AN/AN/ANo defects w ere found during Reg ression Te sting of v ersion 1.0 .N/AN/AN/A N/AN/ANo d efects wer e found du ring Funct ional Test ing of ver sion 1.0. | |
277 | _161147501 8.bin |
Araxis Merge (but not the data content of this report) is Copyright © 1993-2016 Araxis Ltd (www.araxis.com). All rights reserved.