Data Science Methodology - From Deployment to Feedback

Data Science Methodology

From Deployment to Feedback

Deployment

Case Study : Understand the results

Assimilate knowledge for business: (assimilate : 완전히 이해하다, 동화되다, 흡수하다)

- Practical understanding of the meaning of model results

- Implications of model results for designing intervention actions (implication : 영향, 함축, 암시)

 

Case Study : Gathering application requirements

Application requirements:

- automated, near-real-time risk assessments of CHF inpatients

- easy to use

- automated data preparation and scoring

- up-to-date risk assessment to help clinicians target high-risk patients

 

Case Study : Additional requirements?

Additional requirements:

- training for clinical staff

- tracking/monitoring processes

 

Example 1 : solution deployment

- hospitalization risk for juvenile diabetes patients

 

Example 2 : Solution Deployment

- risk summary report by decision tree model note

Example 3 : Solution deployment

- individual patient risk report

Feedback

From Deployment to Feedback

- once the model is evaluated and the data scientist is confident it will work, it is deployed and put to the ultimate test : actual real-time use in the field

 

Case Study : Assessing model performance

Define review process:

- To measure results of applying the risk model to the CHF patient poulation

- Track patients who received intervention : Actual readmission outomces

- Measure effectiveness of intervention : Compare readmission rates before & after model implementation (intervention : 개입)

 

Case Study : Refinement

Refine Model:

- initial reveiw after the first year of implementation

- based on feedback data and knowledge gained

- participation in intervention program

- possibly incorporate detailed pharmaceutical data orgiinally deferred

- other possible refinements as yet unknown

 

Case Study : Redeployment

- Review and refine intervention actions

- Redeploy : continue modeling modeling, deployment, feedback, and refinement throught the life of the intervention program

 

Storytelling

What role does storytelling play in data analysis?

- storytelling with data is a critical skill for data analysis

- important to tell a clear, concise, and compelling story to convince people to take action

- develop a story for your data set to understand your data better

- find the balance between telling a simple story and conveying the complexities of the data

 

- doesn't matter what information you have if you can't communicatate if effectively to your audience

- the best way to communicate your information is through visuals and telling a story

 

- storytelling is an essential skill set - the last mile in delivery

- the ability to extract value from data and to tell a compelling story with data is critical

 

- storytelling is crucial to data analytics

- stories is how you convey your message

- a compelling story helps your audience resonate with your findings

- people remember stories

- stories help build an emotional connect and drive people to action

 

**

댜양한 사람들이 나와서 스토리텔링과 데이터 분석과의 상관관계?를 얘기해줌

결론은 어쨌든...

분석을 하고 나서 그것을 어떻게 표현하는가가 중요하다는 것!

그래서 데이터 분석가와 스토리텔링은 뗄래야 뗄 수 없는 관계

**

 

Course Summary

From problem to approach

Thinking like a data scientist:

- forming a concrete business or research problem

- collecting and analyzing data

- building a model

- understanding the feedback after model deployment

 

Learned the importance of:

- understanding the question

- picking the most effective analytic approach

 

To working with the data

- determining the data requirements 

- collecting the appropriate data

- understanding the data

- preparing the data for modeling

 

To deriving the answer

 

once the analytic apporach is selected, learn how to derive the answer:

- evaluating and deploying the model

- getting feedback on it

- using that feedback constructively so as to improve the model

Remember that the stages of this methodology are iterative!

 

Applied the concepts using a case study

 

The methodology in a nutshell

- The data science Methodology aims to answer the following 10 questions in this prescribed sequence

From problem to approach:

1. what is the problem that you are trying to solve?

2. How can you use data to answer the question?

Working with the data:

3. What data do you need to answer the questiojn?

4. Where is the data coming from (identify all sources), and how will you get it?

5. is the data that you collected representative of the problem to be solved?

6. what additional work is required to manipulate and work with the data?

Deriving the answer:

7. in what way can the data be visualized to get to the answer that is required?

8. does the model used really answer the initial question, or does it need to be adjusted?

9. can you put the model into practice?

10. can you get constructive feedback into answering the question?

 

Module 3 Summary : From Deployment to Feedback

 

Practice Quiz : From Deployment to Feedback

Q. Which of the following statements is correct about the Feedback stage of the data science methodology?

A. Feedback is essential to the long-term viability of the model

 

Q. Which of the following statements represents the essential characteristics of the data science methodology?

A. Data Science Methodology is a highly iterative process. At any point in the methodology, data scientists can decide to repeat a stage or revisit a prior stage and work forward from that previous stage.

 

Q. For predictive models, a test data set, which is similar to but independent of the training set, is used to determine how well the model predicts outcomes - using a training or test. A test data set happens during which stage in Foundational Data Science Methodology?

A. Model Evaluation