Original Article

Validation of the outcomes tools for urinary incontinence in Nursing Outcomes Classification system and their sensitivities on nursing interventions


  • Dercan Gençbaş
  • Hatice Bebiş
  • Sue Moorhead

Received Date: 29.06.2019 Accepted Date: 04.09.2019 Gulhane Med J 2020;62(1):21-27


There are many scales to measure urinary incontinence (UI). Nursing Outcomes Classification (NOC) scales which include Urinary Continence, Urinary Elimination, Self-Care Toileting, Medication Response, and Tissue integrity: Skin and Mucous Membranes are ideal for use in the nursing process for comprehensive and holistic assessment, with surveys available. For this reason, the purpose of this study is to evaluate the validity of these NOC outcomes and indicators used for UI.


This research is a methodological study. Scope validations were applied and calculated according to Fehring 1987 work model. Total of 55 experts which were the sample of the study rated Fehring as a “senior degree” with a score of 90 according to the experts’ scoring. These weighted scores obtained for NOC indicators were classified as critical, supplemental or excluded.


In the NOC system, 5 NOCs proposed for nursing diagnosis of UI were proposed. These outcomes are; Urinary Continence, Urinary Elimination, Tissue Integrity, Self Care-Toileting, and Medication Response. After the scales were translated into Turkish, the weighted average of the scores was obtained from specialists for the coverage of all 5 NOCs. After getting the experts’ opinion, 79 of the 82 indicators were calculated as critical, 3 of the indicators were calculated as supplemental. All NOCs were identified as valid and usable scales in Turkey.


The five NOCs were verified for the evaluation of the output of individuals who received nursing knowledge of UI and variant types. Recommendations include the testing of NOC outcomes in clinical practice and inclusion in nursing curriculum.

Keywords: Nursing outcomes, content validity, nursing diagnosis, urinary incontinence


Nurses perform the nursing care plan and then assess the executed plan in a systematic process (1). In literature, it is stated that in this systematic process, the results improve their objectivity and accuracy if they use the classification systems while considering a client’s health condition and revealing his/her needs (2). These classification systems provide the coding of the data, inserting them into databases as systematized and certifying the nursing process. In this process, they allow the nurses to give the right decision making, to assess with quantitative data and to select the accurate interventions (3).

Among many of the nursing classification systems that are constituted for international usage, NANDA-I in making diagnoses, Nursing Outcomes Classification (NOC) in assessing severity of the issue and patient outcomes and Nursing Interventions Classification (NIC) for nursing interventions are the most used classification systems (4-7). NANDA-I is used actively in Turkey during both nursing education and practice, but NIC and especially NOC systems are not used during education and execution yet (7).

NOC, developed by Iowa University Research team in 1991, is a standard classification system that allows nurses to assess the patients’ before and aftercare status and changes. First NOCs are published in 1997. NOC consists of 7 domains and 31 classes. Each NOC has a code in the taxonomic structure. Lastly, a total of 540 NOCs were published including 52 new ones in 2018. Each NOC has 5 of Likert type scales. In all of the scales, the least wanted state is indicated by the number 1, the most wanted state is indicated by the number 5. In the scales, the point calculation is done through the total point average. Each scale has a different number of articles in itself. The scales should be used privately and in accordance with the need of the individual (2). Nursing care plans can be constituted with the combination of NANDA-I, NIC and NOC systems. In literature, these care plans are called as NNN linkages (5). The main purpose of this study is to be a model for the hospitals in Turkey and the caregivers in public and to provide the assessment of the nursing outcomes with NOC scales. In accordance with this purpose, it has been thought that doing the validities of some of the assessment scales can be a methodological tool in using these NOCs in the execution.

Urinary incontinence (UI) is the most common problem in the worldwide for among elderly, children, disabled ones, and dependent patients (8-11). Especially, it is very common in the aging process and also included in geriatric syndromes. UI is one of the most common one among these syndromes, frequently seen in nursing homes, living quarters, rehabilitation centers (43% to 77%) where elderly people live together (12,13). Although UI is a non-life threatening condition, it is a common health problem affecting the physical, social, work and educational activities of women and decreasing the quality of life (14). So, nurses have an important role to evaluate and improve UI. They need standard and systematic measurement criteria so that they can implement the right initiative. To measure UI, there are many scales and surveys (15-18). Beside them, NOC scales are ideal to be used in the nursing process for comprehensive and integrative assessment. Comprehensive assessment of UI-diseased individual regarding incontinence allows to indicate the incontinence type and therefore to provide accurate nursing care. For UI, NOC and NIC Linkages to NANDA-I and Clinical Conditions Supporting Critical Reasoning and Quality Care Book (page: 234-238) suggested eight NANDA-I Diagnoses (Urinary Elimination Impaired, Urinary Elimination Readiness for Enhanced, Urinary Retention, UI: Urge, UI: Functional, UI: Over Flow, UI: Stress, UI: Reflex), five NOC outcomes (Urinary Elimination, Self- Care Toileting, Urinary Continence, Medication Response, Tissue integrity: Skin and Mucous Membranes) include 92 indicators and 11 major NIC intervention/35 suggested NIC intervention (5).

Therefore, the purpose of this study was to evaluate the content validity and nursing sensitivity of the five incontinence outcomes included in the 5th edition of the NOC (2).

Research questions

1. Which of the five outcomes is most affected by nursing interventions?

2. Which of the outcomes is important for the nursing interventions?


Ethical issues

To execute the research, 50687469-1491-432-15/1648.4-1348 protocol numbered permission letter was taken from the Gülhane Military Medical Academy Ethics Committee. First of all, the research protocol was explained to each expert. After their written approval, scales were asked to score. Written and verbal consents were obtained before receiving opinions of the experts.

Study design

This study was designed methodologically to validate NOC scales, which can be used in the assessment of the conditions of the patients diagnosed with UI. In this study, it was also assessed by the experts how much the NOC Scales indicators contributed to the healing process after the nursing intervention.


Execution and calculation of extent validations were done according to the Fehring’s work model (19). Expert nurses’ inclusion criteria for the study were determined according to the Fehring’s expertise criteria and the criteria of experts were scored (Table 1) (20-22). Including 9 nurses who were expert in the field of sampling, with master degree in a department related to incontinence (9 points), 24 academician nurses (48 points), 21 clinical nurses who used the nursing classification system for at least one year in the clinic (19 points) and an academician nurse with at least four years of clinical experience in urology (4 points), there were 55 experts in total. According to Fehring’s expert scorings, these 55 experts scored 82 points and assessed as Senior degree (Table 1).

Outcome measures

In the assessment of NOCs that were used in collecting data for nursing diagnoses intended for UI, “NOC and NIC Linkages to NANDA-I and Clinical Conditions: Supporting Critical Thinking and Quality Care” guide, which was published in 2012, was used.

This NOC Scales are named as Urinary Continence, Urinary Elimination, Tissue Integrity, Self care-Toileting, and Medication Response. For the validation of these scales, the necessity/materiality of the NOCs and NOC indicators that were recommended for nursing diagnoses intended for UI and the level of contribution that nursing interventions to be executed provided in the healing of this indicators were asked to 55 experts in 2015. According to the Fehring’s model, it was asked to experts to score each of NOC and NOC indicators for arranged extent validation. For the necessity of these indicators, the scoring was described as: “1=not necessary, 2=slightly necessary, 3=necessary, 4=quite necessary and 5=very necessary”. The scoring of the contribution of nursing interventions on these indicators was also assessed as: “1=no contribution, 2=slight contribution, 3=same contribution with other health personnel, 4=slightly more contribution than other health personnel, 5=full contribution” Before the data were collected, NANDA-1, NOC, NIC and NNN linkages systems’ specifications, contents and their positions in nursing process were explained to the experts.

Statistical Analysis

SPSS 21.0 package program was used in the assessment of the data obtained in the research. The data were analyzed with descriptive statistics. For each NOC and NOC indicators, the scorings were calculated as follows: “1=0 points; 2=0.25 points; 3=0.50 points; 4=0.75 points; 5=1.00 point”. After getting the expert opinion, these weighted points obtained for each NOC and NOC indicator were classified as “critical in >0.8 and supplemental in 0.8>, <0.5 and >0.5” was removed (23).

To assess the internal consistency between expert opinions, Cronbach alpha factors of all articles were assessed. If Cronbach alpha factor was between 0.00 and 0.39, then the test was assessed as not reliable, if between 0.40 and 0.59, then as quite reliable and if between 0.80 and 1.00, then as highly reliable (24).


In NANDA-I system, in the third field named “elimination and exchange” and in the “urinary function” class, the nursing diagnoses of UI were published; 00019 urge UI, 00017 stress UI, 00018 reflex UI, 00020 functional UI, 00176 overflow UI, 00016 impaired urinary elimination, 00166 readiness for enhanced urinary elimination, 00022 risk for urge UI, 00023 urinary retention. 5 NOCs were recommended for these diagnoses. Extent validations of these 5 NOCs and weighted average of the expert scoring for the contribution of nursing interventions raised above 0.8 (Table 2).

Urinary continence outcome was defined as “control of elimination of urine from the bladder” and it consisted of 19 indicators (2). It evaluated the urinary continence with individual’s responding to urge timely manner, void inappropriate receptacle, start and stop the stream, managing clothing independently, urine leakage between voiding, with increased abdominal pressure, wets clothing and urinary tract infection.

Two of them (“Voids >150 milliliters each time” and “Manages clothing independently”) were classified as “supplemental” and others as “critical”. Besides, for average scoring taken after expert opinion regarding the contribution of “Manages clothing independently” indicator to interventions was calculated as 0.77, and as below 0.8, this indicator was identified as a “supplemental” indicator intended to contribute to nursing interventions (Table 3).

“Urinary Elimination” outcome was defined as “collection and discharge of urine” and consisted of 21 indicators (2). This NOC evaluated the urinary elimination with indicators which were elimination pattern, urine features as odor, amount, color, clarity, fluid intake status, the individual’s ability to completely empty the bladder, types of incontinence as stress, urge, functional, urinary status as frequency, urgency, retention, nocturia, pain. All of the indicators in Urinary Elimination were classified as “critical” according to the experts’ opinion.

“Tissue Integrity: Skin and Mucous Membranes” outcome was defined as “Structural intactness and normal physiological function of skin and mucous membranes” and it consisted of 22 indicators (2). Only “Hair growth on skin” indicator was classified as “supplemental” regarding its necessity and contribution to interventions. The other indicators were skin features as temperature, sensation, elasticity, hydration, perspiration, texture, thickness, integrity, tissue perfusion, hair growth on skin and abnormal pigment features as lesions, mucous membrane lesions, scar, erythema, blanching, necrosis, induration, corneal abrasion, skin cancers, flaking, scaling. All of them were evaluated as “critical”.

“Self-Care-Toileting” outcome was defined as “Personal actions to toilet self independently with or without assistive device” and it consisted of 13 indicators (2). All of the indicators were classified as “critical” for the experts’ opinion. They were patient’s response to full bladder, response to urge to bowel movement in timely manner, patient’s ability to get in and out of bathroom, to remove clothing, position seat on toilet or commode, to get toilet between urge and passage of urine, to get toilet between urge and evacuation of stool, to empty bladder and bowel, to wipe self after urinating and bowel movement, to get up from toilet or commode, to adjust clothing after to toileting.

“Medication Response” outcome was defined as “Therapeutic and adverse effects of prescribed medication” and it consisted of 10 indicators (2). All of the indicators in Self-care-Toileting outcome were classified as “critical” for the experts’ opinion. They were therapeutic effects, change in blood chemistries and symptoms, behavioral, maintenance of expected blood levels allergic reaction, adverse effects, medication interactions, medication intolerance, and adverse behavioral effects.

Cronbach alpha factor calculated for internal consistency of the scorings given by the experts (n=55) was found to be above 0.80 (0.873-0.959) for 5 NOCs (Table 4).

As a result, 79 of the indicators were evaluated as critical and 3 of indicators as supplemental after expert opinion. These three indicators evaluated as supplemental are shown in Table 2. Besides, none of the articles was removed for not obtaining any scoring as 0.5>. All of NOC outcomes were identified as valid and usable scales in Turkey.


All of 5 NOC outcomes were assessed as “critical” by the experts. These results have shown the importance of analyzing of the continence status, the urinary excretion status, the existence of skin lesions and the effects of self-care and medicine used in the nurses’ assessment of individuals regarding UI. These outcomes can be used by the nurses to obtain comprehensive and standard data in the assessment of the individual receiving nursing diagnosis of UI.

The indicators in the urinary continence outcome that assess the continence status of an individual like one’s knowing of toilet need sense, not holding the urine after toilet need sense, urinary tract infection status, daily amount of liquid taken, knowing of the medicine that spoils urine control, going to toilet independently, urine leaking ways (as coughing, as sneezing, etc.) were assessed as critical. In the researches in which urinary continence status of individuals are assessed, not all but similar outcomes as incontinence type, medical drugs which effect urinary elimination, mobilizing, the status of infection have been questioned (23-26). “Voids >150 milliliters each time” and “Manages clothing independently” indicators’ being assessed as “supplemental” by the experts may be for their thoughts of them being valid for elders and children and not being the indicators that appeal to the general population. Urinating more than 150 milliliters and addiction level have been emphasized as important factors in assessing the individual regarding UI (27,28).

The assessment of all of urinary elimination indicators as critical by the expert nurses shows us the importance of the assessment of each factor that fazes urinary tract infection, in parallel with the studies that assess the effectiveness of nursing interventions executed to UI-diseased individual (23,25).

In literature, there are study reports regarding the importance of perineal skin assessment of UI-diseased individuals, especially the ones using the diaper, and the big contribution of this assessment to selecting nursing interventions (29,30). In this study, the experts consider the indicators like skin temperature, sensation, elasticity, perspiration, skin integrity, skin cancer, skin lesion, skin scaling and necrosis valid as the indicators important to the assessment of skin integrity. However, in contrast to the literature, the indicator “Hair growth on skin” was assessed as supplemental. This case may be arisen from its being thought on the basis of children and this indicator not being considered as important as others for there not being hair growth on perineal skin in children.

The indicator of the Self-care-Toileting includes the relationship between people’s incontinence status and self-care status (taking off the clothes before-after urinating, providing himself hygiene after urinating, addiction status, etc.) Experts accepted all indicators in Self-care-Toileting outcome valid as “critical” and stated that the assessment of these indicators provided a big contribution in selecting nursing interventions. In 2012, as a result of 5th International Consultation on Incontinence in Paris, it was stated that it was necessary the individuals be assessed in terms of self-care related to toileting because of peripheral arrangement and self-care being important interventions in incontinence management (31).

The indicators of the outcome of Medication Response provide to assess the medicine’s healing or side effects seen on people as both behavioral and blood findings. Expert nurses found all articles of this NOC important as “critical”. In literature, it is stated that some medicine used by patients, especially by elders, may lead to incontinence and it is emphasized that their pharmacological treatment and their effects should definitely be assessed (9).

In the light of these results, we can say that NOC scales are a valid tool to assess people regarding UI. We can say that the consistency between experts is high because of nearly all of the obtained scorings being very close to each other. Using them in nursing care plans will lead to comprehensive and accurate interventions as a result of the assessment with more standard and valid indicators and provide the chance for the assessment of intervention’s effectiveness again with standard and valid indicators. In literature, NOC scales have been stated as an active assessment tool in the assessment of nursing diagnosis’ degree in NNN linkages and in the assessment of activeness of executed interventions (32-34).

From the nurses who had at least 4 years of clinical experience in urology, there was only 1 nurse who made a return. If this number had been higher, a higher expert scoring would have been provided and an assessment of NOC outcomes with a more comprehensive overview would have been performed. Despite this, by keeping the number of our other experts high, scoring was held above 20 and Senior degree was reached.


In this study, five NOC outcomes were verified to assess the outcomes of people receiving nursing diagnosis of UI and its various types. It was also stated that these five NOC outcomes contributed to deciding on nursing interventions.

All 82 indicators were approved as valid. Seventy-nine of indicators were assessed as critical and 3 of indicators were assessed as supplemental according to the scoring from experts to verify the indicators. At the same time, the contribution to planning nursing interventions of 2 of the indicators that were assessed as supplemental was assessed as supplemental.

Despite NANDA-I’s being executed by students and in hospitals in our country, NOC and NIC were being lectured only as a theoretical lesson. With this study, an awareness has been created for using an international and standard nursing assessment system that can be used in the assessment of people’s health issues or risks.

As a suggestion for future studies, we recommend using these NOC outcomes in execution after they are repeatedly used on people who are from different age groups and have different diseases.


Ethics Committee Approval: To execute the research, 50687469-1491-432-15/1648.4-1348 protocol numbered permission letter was taken from the Gülhane Military Medical Academy Ethics Committee.

Informed Consent: Written consents were obtained before receiving opinions of the experts.

Peer-review: Externally peer-reviewed.

Authorship Contributions

Concept: D.G., H.B., Design: D.G., H.B., Data Collection or Processing: D.G., Analysis or Interpretation: D.G., Literature Search: H.B., S.M., Writing: D.G., H.B.

Conflict of Interest: No conflict of interest was declared by the authors.

Financial Disclosure: The authors declared that this study received no financial support.

  1. Meleis AF. Theoretical Nursing Development & Progress. 5th ed. China: Wolters Kluwer Heath/Lippincott Williams&Wilkins; 2012.
  2. Moorhead S, Swanson E, Johnson M, Maas M. Nursing outcomes classification (NOC): Measurement of Health Outcomes. 6th ed. St. Louis MO: Elsevier; 2018.
  3. Henry SB, Mead CN. Nursing Classification Systems: Necessary but not Sufficient for Representing ‘‘What Nurses Do’’ for Inclusion in Computer-based Patient Record Systems. J Am Med Inform Assoc. 1997;4:222-232.
  4. Herdman TH, Kamitsuru S. NANDA International nursing diagnoses: definitions and classification, 2018-2020. 11th ed. Thieme;2017.
  5. Johnson M, Moorhead S, Bulechek G, Butcher H, Maas M, Swanson E. NOC and NIC Linkages to NANDA-I and Clinical Conditions. Supporting Clinical Reasoning and Quality Care. 3rd ed. MO:Mosby:2012.
  6. Butcher HK, Bulechek GM, Dochterman JM, Wagner CM. Nursing Interventions Classification (NIC). 7th ed. St. Louis: MO:Elsevier;2018.
  7. Tastan S, Linch GC, Keenan GM, et al. Evidence for the existing American Nurses Association-recognized standardized nursing terminologies: A systematic review. Int J Nurs Stud. 2014;51:1160-1170.
  8. Kuijper C, Chrzan R. Editorial: Urinary Incontinence in Children: Controversies Concerning the Bladder Outlet. Front Pediatr. 2018;6:216.
  9. Ouslander JG. Geriatric urinary incontinence - Reflections on a EUGMS consensus update. European Geriatric Medicine. 2017;8:197-199.
  10. Ayeleke RO, Hay-Smith EJ, Omar MI. Pelvic floor muscle training added to another active treatment versus the same active treatment alone for urinary incontinence in women. Cochrane Database Syst Rev. 2013;20:CD010551.
  11. Abrams P, Cardozo L, Fall M, et al. The standardisation of terminology of lower urinary tract function: Report from the Standardisation Sub-Committe of the International Continence Society. Neurourol Urodyn. 2002;21:167-178.
  12. Booth J, Hagen S, McClurg D, et al. A feasibility study of transcutaneous posterior tibial nerve stimulation for bladder and bowel dysfunction in elderly adults in residential care. J Am Med Dir Assoc. 2013;14:270-274.
  13. Vinsnes AG, Helbostad JL, Nyronning S, Harkless GE, Granbo R, Seim A. Effect of physical training on urinary incontinence: a randomized parallel group trial in nursing homes. Clin Interv Aging. 2012;7:45-50.
  14. Özcan M, Kapucu S. Nursing Approach in Elderly Patients with Urinary Incontinence. Hacettepe University Journal of Nursing Faculty. 2014;1:101-109.
  15. Shumaker SA, Wyman JF, Uebersax JS, McClish D, Fantl JA. Health-related quality of life measures for women with urinary incontinence: the Incontinence Impact Questionnaire and the Urogenital Distress Inventory. Continence Program in Women (CPW) Research Group. Qual Life Res. 1994;3:291-306.
  16. Uebersax JS, Wyman JF, Shumaker SA, McClish DK, Fantl JA. Short Forms to Assess Life Quality and Symptom Distress for Urinary Incontinence in Women: The Incontinence Impact Questionnaire and The Urogenital Distress Inventory. Continence Program for Women Research Group. Neurourol Urodyn. 1995;14:131-139.
  17. Wagner T, Patrick DL, Bavendam TG, Martin ML, Buesching DP. Quality of life of persons with urinary incontinence: development of a new measure. Urology. 1996;47:67-71.
  18. Sandvik H, Hunskaar S, Seim A, Hermstad R, Vanvik A, Bratt H. Validation of a severity index in female urinary incontinence and its implementation in an epidemiological survey. J Epidemiol Community Health. 1993;47:497-499.
  19. Fehring RJ. Methods to validate nursing diagnoses. Heart Lung. 1987;16:625-629.
  20. Quatrini Carvalho Passos Guimarães HC, Pena SB, Lopes Jde L, Lopes CT, Bottura Leite de Barros AL. Experts for Validation Studies in Nursing: New Proposal and Selection Criteria. Int J Nurs Knowl. 2016;27:130-135.
  21. de Fátima Lucena A, Holsbach I, Pruinelli L, Serdotte Freitas Cardoso A, Schroeder Mello B. Brazilian Validation of the Nursing Outcomes for Acute Pain. Int J Nurs Knowl. 2013;24:54-58.
  22. Seganfredo DH, Almeida Mde A. Nursing outcomes validation according to Nursing Outcomes Classification (NOC) for clinical, surgical and critical patients. Rev Lat Am Enfermagem. 2010;19:34-41.
  23. Borrie MJ, Bawden M, Speechley M, Kloseck M. Interventions led by nurse continence advisers in the management of urinary continence: a randomized controlled trial. CMAJ. 2002;166:1267-1273.
  24. Alpar R. Applied Statistics and Validity-Reliability. 2nd ed. Detay: Ankara; 2012:497-509.
  25. Klay M, Marfyak K. Use of a continence nurse specialist in an extended care facility. Urol Nurs. 2005;25:101-102.
  26. Aslan E, Komurcu N, Beji NK, Yalcin O. Bladder training and kegel exercises for women with urinary complaints living in a rest home. Gerontology. 2008;54:224-231.
  27. Palese A. Carniel G. The effects of a multi-intervention incontinence care program on clinical, economic, and environmental outcomes. J Wound Ostomy Continence Nurs. 2011;38:177-183.
  28. Sackley CM, Rodriguez NA, van den Berg M, et al. A Phase II Exploratory Cluster Randomized Controlled Trial of a Group Mobility Training and Staff Education Intervention to Promote Urinary Continence in UK Care Homes. Clin Rehabil. 2008;22:714-721.
  29. Al-Samarrai NR, Uman GC, Al-Samarrai T, Alessi CA. Introducing a new incontinence management system for nursing home residents. J Am Med Dir Assoc. 2007;8:253-261.
  30. Thompson P, Langemo D, Anderson J, Hanson D, Hunter S. Skin care protocols for pressure ulcers and incontinence in long-term care: A quasi-experimental study. Adv Skin Wound Care. 2005;18:422-429.
  31. Wagg A, Gibson W, Ostaszkiewicz J, et al. Urinary incontinence in frail elderly persons: Report from the 5th International Consultation on Incontinence. Neurourol Urodyn. 2015;34:398-406.
  32. Azzolin K, Mussi CM, Ruschel KB, de Souza EN, de Fátima Lucena A, Rabelo-Silva ER. Effectiveness of nursing interventions in heart failure patients in home care using NANDA-I, NIC, and NOC. Applied Nurs Res. 2013;26:239-244.
  33. Kautz DD, Van Horn ER. An exemplar of the use of NNN language in developing evidence-based practice guidelines. Int J Nurs Terminol Classif. 2008;19:14-19.
  34. Park H. Identifying core NANDA-I nursing diagnoses, NIC interventions, NOC outcomes, and NNN linkages for heart failure. Int J Nurs Knowl. 2014;25:30-38.