Prática de Shadowing: Decision-Making and Entrepreneurial Performance: Evidence from a Field Experiment - Aprenda a falar inglês com vídeo

thank you thank you very much ryan can you hear me and can you see the slides perfect

perfect thank you so much so i'm elena novelli from base business school

and as you mentioned today i'm presenting we are presenting together with chiara from inside this paper

and we would like to first of all thank sms for organizing this session we are delighted to be here

and also i can see there are many friends in the audience

so thank you

so much for taking the time to attend our presentation we

are really at a stage where we would certainly benefit from feedback

and comments so we look forward to hearing from you later

so this paper builds on the observation that innovation settings

and entrepreneurial settings in particular are characterized by a high degree of uncertainty

so part of which is endogenous to firm action

so a very important question in these settings is how can firms navigate this uncertainty

and in particular what kind of decision -making processes can help them navigate uncertainty.

So if we look at prior research there are essentially two main approaches

that have been identified that can help firms navigate uncertainty.

First of all what we can call a cognitive -based approach.

A cognitive -based approach is essentially based on the idea of addressing uncertainty by reasoning through theory,

frameworks or rules that can help decision makers understand what to do.

Alternatively, decision makers can use what we could call evidence -based approaches.

These approaches are essentially based on the idea of addressing uncertainty via the systematic collection of evidence and its discipline assessment,

with the idea that in this way they can learn from the environment.

Now, more recent evidence has suggested that there are actually important synergies that exist between cognitive and evidence -based approaches.

For example, the work of Cathy Eisenhardt

and Chris Bingham suggests about the importance of combining thinking and doing to devise superior strategies.

Or, for example, the work of Rory McDonald and Cathy Eisenhardt shows that entrepreneurs who combine cognition with evidence,

for example, using experimentation to test the key assumptions underlying a business model, can accelerate their learning.

Or finally, the work of Camufo and colleagues shows the benefits of a scientific approach to decision making,

an approach that combines elements of cognition, as well as elements of experimentation.

So we use the terminology used by these authors,

and we will talk about the scientific approach to decision -making to refer to an approach that uses theory,

hypothesis, but then tests these hypotheses with evidence and conducts a discipline evaluation of that evidence.

The expression scientific approach refers to the fact that approach that uses theory,

hypothesis, evidence and evaluation really resembles that used by scientists in the development of knowledge.

Now, let me give you an example about how such an approach might actually work in practice, especially in the context of entrepreneurial decision making.

So here is an example of an entrepreneur we worked with.

Essentially, this entrepreneur identified and developed a theory about the fact

that there is actually some unmet demand in electronics that comes from non -tech savvy consumers.

These consumers are really more interested in the usability of products, in the aesthetics of products.

They don't really know the details about the technical specifications.

So they know what they want to achieve, but they don't know exactly what technical specifications they need to achieve that.

So this entrepreneur developed a theory about these types of consumers

and how their demands could be met and developed a set of hypotheses.

An example of one of these hypotheses was,

for example, that providing customers with information on product compatibility was more relevant

when they were making a purchase than providing them with very detailed technical specifications

so they devised a set of tests to for their hypothesis

and for example in this case they ran an online test

and a b tests for two weeks offering customers different product

specifications on the one hand information about product compatibility on the

other hand information about technical specifications of products they found

that their hypothesis was supported and so

So they continued iteratively to test hypothesis about their theory until

they reached the stage where they developed a very solid and robust value proposition that enabled them to grow and succeed. So...

Let me synthesise what a scientific entrepreneur is.

A scientific entrepreneur is one who starts with a mental model that explains how a business creates value.

He or she then proceeds to articulate, to decompose that theory into testable propositions that logically flow from the theory.

They then perform rigorous tests with representative samples of customers, and then they assess the evidence collected in a disciplined way.

If this is a scientific entrepreneur, who is then a non -scientific entrepreneur?

A non -scientific entrepreneur is one who draws conclusions based on testing, but with no counterfactuals.

It's someone who tests multiple versions of a product, but without being driven by a theory.

Therefore, there is a higher degree of causal ambiguity.

Or someone who performs lower quality tests, for example, multiple versions of a product or a service are tested, but they differ more than one characteristics.

So the signal that those tests can reveal is certainly more noisy.

So the key research question that this paper addresses is to what extent

and under what conditions is a scientific approach to decision -making,

so an approach that combines cognition and evidence, associated with superior performance.

And in particular, what we want to understand is whether a scientific approach can benefit all firms equally.

Now, we believe that this is a very important question, especially because the main evidence

that has identified a positive relationship more in general between systematic decision -making approaches

and performance tends to be focused on more established firms.

So we want to understand this question more closely.

Thanks, Elena.

So this leads us to the focus of the paper, which is how do we expect the scientific approach to impact the performance of entrepreneurial firms and of different firms in particular?

So we're going to look at the impact of a scientific approach on performance first.

If you could please go to the next slide.

And we have a baseline hypothesis.

What we are expecting building on literature on strategy, entrepreneurship, decision -making, and organizational learning is that overall,

the scientific approach will improve this process of iteration that entrepreneurs usually go through.

This will result in better decisions and in better performance.

And we expect this because we are considering what the scientific approach does as entrepreneurs are making decisions.

On the one hand, as Elena was describing, the scientific approach is very beneficial

because entrepreneurs have a mental model they have a theory

and this implies that they have a clear representation of the problem they're facing and they're then able to articulate hypotheses

what theory and hypotheses do is

that they lead to more precise tests they're basically spotlighting what are the key assumptions

that entrepreneurs have and that they need to test in order to make sure

that a business is viable now this increased precision results in tests

which are more focused and therefore are more likely to result in learning in addition to

that the entrepreneurs are also gathering evidence

that they are evaluating in a very systematic manner and

so what evaluation does is

that it helps entrepreneurs first of all understand whether the evidence is supporting their initial assumptions

but also it's very beneficial when entrepreneurs do not find evidence actually support their hypothesis

which happens quite frequently and when that is the case the fact

that entrepreneurs have a clear framework is actually very beneficial in helping them in understanding what to do next

so for these reasons we're expecting

that the scientific approach will have a positive effect on decision making

and performance now we are also interested as we mentioned earlier in understanding what happens with firms

100

that are different levels of business development and what do we mean by that

101

we have looked at the literature and interestingly

102

when it comes to entrepreneurship entrepreneurship Entrepreneurship is often thought about as a journey

103

that requires firms to go through various steps and go through an evolution process.

104

So you can think of an entrepreneurial firm as some scholars have used the parallel of the human life.

105

So when a firm is born, like a baby, it's not fully formed.

106

It's in a very embryonic stage, if you will, which we can consider the startup phase, which is also described as a progressing nascent venture.

107

But this can progress over time, going all the way to a more established phase where the firm becomes a more sustainable and continued market participant.

108

Now, the reason why this level of business development, and this is the construct that we're focusing on here, is particularly important is because these firms are in fact facing different problems.

109

The fact that the firm is more or less developed actually implies that they've conducted slightly different activities, but they're facing different problems.

110

And the fact that they're facing different problems has important repercussions for the type of decisions that they're making.

111

to make this a little bit more concrete let me provide some definitions and also give an example

112

when we are looking at firms that are lower level of development

113

and remember that this concept is a continuum so the example

114

that we have here is just a simplification

115

so we're providing an example of a low and high level

116

but of course there is a continuum you can think of

117

a firm with a low level of development as a firm

118

that is still quite early on in the process of venture creation and

119

so doesn't have a particularly well -defined value proposition just yet

120

on the other hand of the spectrum you can have a firm

121

which is at a higher level of business development meaning

122

that it's later on in the stage of development as a firm

123

and this means that the value proposition is more defined and

124

so the entrepreneur is more fine -tuning this value proposition now to be very specific let's look at an example

125

and the example that we have here is one simple business idea help busy individuals stay fit

126

so these are people who usually don't have time to go to the gym

127

or attend some classes so you can have an entrepreneur

128

that is trying to develop this business idea and here we have images

129

that represent these two different levels of development on the left

130

hand side what you are seeing is the low level of

131

development where the entrepreneur has not decided yet how she's going

132

to deliver this value to the customer this could happen through sessions

133

which are close to for instance a park where the person lives they could be delivered online

134

or they could be offered through a gym in a van on the right hand side what you're seeing is

135

that the entrepreneur instead has decided

136

that the offer will happen through a gym in a van

137

and so this gives you a sense of what it means to have a more

138

or less developed business idea now what this does is

139

that it implies

140

that these firms as we were talking earlier about the level of development are facing different problems And in particular, when we're looking at firms which have a lower level of business development,

141

they're facing broader problem.

142

By their very own nature, these problems are more open, they're more complex, there are more aspects that need to be taken into account.

143

So in this case, what we are expecting, building on literature on organizational learning in particular, is that there will be more ambiguous learning and a slower pace of feedback,

144

which is something that usually happens when actors are dealing with problems, actions, and consequences, which are quite broad in scope.

145

As a consequence, this implies that it's going to be harder for these entrepreneurs to learn

146

because the information that are emerging will be noisier.

147

On the other side, when we're looking at a higher level of business development instead, what happens is that these firms have different types of problems

148

that they're facing they are narrow in scope and what happens

149

when actors are facing narrow problems is

150

that the process of decision making is simplified and this implies

151

that it will be easier for these entrepreneurs to apply the scientific approach

152

and here you can think of some studies like eti raj

153

and levin tall but also philip xsr that show how

154

when you simplify this landscape of decision making for instance by keeping some aspects of decision making constant

155

and just varying some aspects of the problems to change then these results in simplified processes of decision making

156

and more effective decision making so the argument is

157

that there will be enhanced learning for these firms

158

which are the more advanced level of business development and

159

so for these reasons we're expecting

160

that the level of business development is moderating positively the impact of a scientific approach on performance

161

how do we test these ideas well we ran a field

162

experiment as it was alluded to earlier with the title

163

and we use actually business support program to run this experiment

164

you can see some pictures here from the study you can

165

see elena myself the entrepreneurs the program were broadly it was a project

166

that was made possible thanks to generous funding from the uk government

167

which allowed us to create a program just for the purpose of this particular study

168

and this meant

169

that we recruited research assistants instructors in we created a demo

170

day with investors i'm going to provide a little bit more details about

171

that but most importantly we have firms

172

which are at various level of business development participating to the

173

study here's our review of the rct uh is i'm just

174

going to provide some high level details of the study of course there are a lot of details with experiments, so feel free to ask any questions that you might have.

175

What we did is that we started off with a call for applications.

176

So we created a website and we started a marketing campaign.

177

We managed to recruit 261 firms.

178

After that, we randomly allocated them to either treatment or control condition.

179

And what's important to point out here is that they both attended training.

180

Both groups received the same amount of training for the same amount of hours with the same instructors on the same topics.

181

The only difference is that 3D entrepreneurs learned about a scientific approach and entrepreneurs in the control group didn't.

182

As a final step, we conducted data on these entrepreneurs for about 10 months using some phone interviews, as we're going to show you more details.

183

This is a slide that gives you a sense of the timeline of the project.

184

The project overall from design to end lasted for 12 months, but here what you can see is that we started off the training in month one,

185

which was February 2019, and we ended in April 2019.

186

During this time, we had 21 hours of training split over seven sessions, and this is where the treatment was.

187

After that, until December 2019, we had monthly events.

188

The monthly events were exactly the same for the treatment and the control group.

189

We actually ran the same event twice on different days

190

or different times of the day to prevent these people from talking and potentially disclosing elements of the treatment.

191

As you can see, the data collection goes on from right before the treatment started all the way until the end, which was December 2019.

192

to give you a little bit more details about the participants what we did as i mentioned we recruited firms

193

that were at different levels of business development there were no prerequisites in terms of industry

194

so we ended up with a beautiful variety in terms of the type of businesses

195

that attended the program from those that were interested in fashion retail fintech

196

security environment education really different type of businesses what they all

197

had in common however was the size of the firm they all the firms could have at most 10 employees

198

and the reason why we made that particular choice is

199

that we actually wanted whoever participated into the program to be

200

heavily involved in the decision making process of the firm

201

and this condition is more accurately met in the context of these small firms the table

202

that you're seeing on this slide is showing the balance checks

203

so as i mentioned after recruiting these entrepreneurs we collected some data on the firm

204

and then we use this to check whether the randomization actually resulted in balanced groups

205

and what this table is showing you is

206

that there are no statistically meaningful differences on a range of covariates

207

which is looking at both aspects which are more demographic

208

when it comes to the firm such as what's the age the team size what's the gender ratio within the company,

209

to some which have more to do with decision making and performance.

210

To give you some details about the treatment versus the control condition,

211

as I mentioned earlier, we use the training in the same way for treatment and control firms with one important difference.

212

You can see that both the control and the treatment group are exposed to strategy frameworks, for instance, the business model canvas or balance scorecards,

213

and also data gathering techniques.

214

And so this includes, for instance, how do you run a survey?

215

How do you conduct A -B testing?

216

How do you talk to your customers?

217

How do you conduct good customer interviews?

218

The only difference is that the treatment group was also taught for each of these topics

219

and each of these sessions to use a scientific approach to decision -making.

220

And so what this meant is that for each step, they had to have a very clear theory of a mental model that explain how their business generates value.

221

They had to then develop hypotheses or testable propositions to test what their key assumptions were, and then conduct tests in a rigorous manner and evaluate the results.

222

So to give you a better sense of how we did this in practice, the classes were highly experiential.

223

So we had usually a fairly long lecture, so it was around one hour, one and a half hour.

224

We had various in -class activities to really make sure

225

that the entrepreneurs would understand the content that was explained to them right away

226

and we got them to apply the content of the course to their business right away

227

so the session also included some time for the entrepreneur to apply this content to their business

228

and their business idea while they were in class

229

and clarify with instructor whether they have any questions in the

230

case of the first session where we teach the business model canvas for instance what we did is

231

that we teach the business model canvas to both groups they

232

conduct the same activities however there is a difference in how they're taught to use the business model canvas.

233

control group follows a more standard approach

234

which is used in a lot of universities and accelerators

235

and business schools as well where they learn to use the

236

business model canvas as a representation of their business where various elements should be connected between them

237

and it contains some elements

238

that will be tested later on in the case of the

239

scientific approach they still learn about the business model canvas however they learned

240

that that represents the basis for their theory of their firm

241

and so they are asked to also

242

and taught how to write a theory from their business model canvas

243

and they consider each element of the business model canvas as hypothesis

244

that will be tested at a later stage

245

and then we follow this logic

246

and implement similar changes throughout the course to make sure

247

that the content remains the same but the only difference is the scientific approach

248

and uh to give you also a little bit more details about the data collection process what we did is

249

that we follow an approach which was championed by bloom

250

and renan in economics with their study of management practices where

251

we decided to conduct telephone interviews with the cell phone interviews we had a team of research assistants

252

that were extensively trained on a predefined interview script.

253

All interviews were recorded and this was helpful for us to make sure that we could provide feedback to the research assistants.

254

And all these interviews ran for about once a month from February until December 2019.

255

And the research assistants were asking entrepreneurs about various topics.

256

There were both some close -ended questions

257

that were primarily measuring performance such as how many revenue did

258

your business generate in the last month what kind of cost

259

did you make any changes to your business as we're going

260

to see in in the next few slides with the results

261

and then there were some more open -ended questions as well

262

which aimed at measuring more the decision -making process that the entrepreneurs were going through

263

i think this is the end of the design part so i can leave the floor to elena for the results

264

Thank you.

265

So let me give you an overview of what we find.

266

So the first question that we wanted to address is whether treated entrepreneurs achieve better performance.

267

So when it comes to estimating performance,

268

prior research has extensively documented that firm performance is not a unidimensional construct

269

and that difference performance measures really capture different nuances of performance

270

so when we look at performance we look first of all at revenue as an established measure of performance

271

but we also look at firms growth and

272

because we know from private work that some firms focus on growth first

273

because by doing this they basically go on a better performance trajectory

274

so we wanted to capture both effects and

275

so we wanted to capture both dimensions

276

and therefore we look at both our analysis follows a classic

277

difference in different specification we therefore estimate our performance variable as a function of treatment post

278

which is an indicator variable that is valued as one for firms after the training

279

and the interaction between the two so here what we see is

280

that when we look at size of firms actually uh what we find is

281

that the treatment had a positive effect on perform on performance

282

as measure as growth in particular about a 12 % increase on performance.

283

However, interestingly, if we look at revenue, we see a non -significant effect.

284

Now, in looking at these results, we must acknowledge that our observation window was quite limited,

285

about 10 months, which is not necessarily a huge amount of time for a firm to observe a change in performance.

286

Now, ideally, we would want to observe a longer time horizon

287

that this was not possible at the moment due to obviously finding funding constraints.

288

However, to the extent that, as prior research has suggested, growth can be interpreted as an early performance measure,

289

we could interpret this result to indicate that firms

290

that are treated are led to invest in the size of the firm by hiring more employees

291

and investing in the business model

292

and this might later on lead them to achieve a superior performance in terms of revenue as well.

293

But we will go back to this point as we see more results.

294

We move next to try and understand the extent to which the scientific approach benefits all firms equally.

295

So what we do here, we use a triple difference specification, so we want to see whether the stated difference

296

that we observed between the performance before and after the treatment for treated and controlled firms,

297

how it is shaped by our

298

key variable which is the degree of business development so what we see here was that

299

if we look at different types of firms a different degree of development

300

if we look at firms more developed the treatment had an additional incremental effect on their size so

301

if we observed a impact of the treatment on the size

302

of the firm of about 11 percent what we see here is

303

that one standard deviation increase in the degree of business development

304

leads to an additional three percent increase in the size of

305

the firm what is extremely interesting for us is to observe what happens

306

when we look at revenue so you will remember

307

that we didn't find a significant effect of the treatment on performance as measured in terms of revenue.

308

Now, if we make now a distinction between firms at a low level of business development

309

and firms at a high level of business development, what we see is really interesting.

310

We see that for firms at a very low level of business development, the treatment actually had a negative effect on revenue.

311

In particular, a negative effect of about 30%.

312

On the other hand, if we look at the case of firms with a high degree of business development,

313

the treatment had a positive effect. In particular,

314

one standard deviation increase in the degree of business development is associated with a substantial increase in the performances measured in revenue,

315

a 60 % increase, which is obviously quite substantial.

316

Now, the way we interpret these results is the following.

317

We interpret these results to basically indicate that for less established firms, the method brings them back to the drawing board.

318

And so they start using the method to work again on their business model, to work again on their value proposition,

319

to change things, to do things perhaps differently.

320

And essentially, we see these being represented by the fact that they invest in their company by growing the number of employees.

321

However, they are not yet at a stage, an exploitation stage, where we see an impact on revenue.

322

On the other hand, if we look at more established firms, these are at a stage where they are already in an exploitation phase in their business model.

323

So they use the method differently.

324

They use the method to fine tune their value proposition

325

and make even more revenue and make it even and make it basically even more successful.

326

Now, this is very much in line with the findings of other scholars.

327

For example, if we think about the work of Joshua Gantz,

328

Scott Stern and colleagues, they suggest in line with these results that when firms are still in an early low commitment phase,

329

any positive signal leads them to an exploration search phase.

330

While when firms have already made significant commitment, a positive signal leads them to higher exploitation.

331

Now, these results are also robust to winsorizing alternative measures of development and fixed effect specification.

332

In a way, looking at comparing what happens to firms at a low degree of business development

333

and firms at a high degree of business development helps us

334

perhaps providing an answer to our earlier question about what happens in the longer term,

335

because what we see here is a comparison between firms that are at a different stage of the journey.

336

So this gives us a sort of snapshot at different points in time, which might suggest that in the longer term,

337

the method also helps firms exploiting and benefiting in terms of revenue.

338

More in general, we are also in conversation with the government to explore the

339

possibility of observing these firms in the longer term by basically having a longer time horizon,

340

by merging our data with the data collected by the government.

341

So this would answer our question regarding longer term even more extensively of course meanwhile we had a relevant shock

342

which was of course the pandemic

343

but this might also offer some the interesting opportunity to see whether the methods made these firms for example more resilient

344

Let me show you also some model free evidence

345

that might give us another snapshot of what happens if we look at the data.

346

So what we have here on the left hand side is

347

the average of revenue at the baseline for firms at a low degree of development,

348

lower than the average, and a high degree of development, higher than the average.

349

And we compare control and treated firms.

350

So what we see here is

351

that the revenue of firms at a low degree of development were very similar in the control and in the treatment group.

352

And also the revenue of firms at a high degree of development were very similar for the control and treated firms.

353

If we observe the averages of the revenue at the end of the period, we basically see that, of course,

354

all the revenue of all firms grew, but in the low development group,

355

firms in the control group grew more in terms of revenue compared to treated firms,

356

while the opposite is true in the case of firms at a higher degree of development.

357

So this provides us another snapshot of the results that we showed in our econometric analysis.

358

Now we are at a stage where we are trying to provide more evidence

359

about the mechanism that we suggest are behind our results and that we mentioned earlier.

360

So what we have started looking at is we asked the entrepreneur before starting our program, what were the issues that they faced?

361

And what we see is that the issues

362

that entrepreneurs at different levels of business development face are indeed different as predicted by our theory. In particular,

363

we've seen that firms at low levels of business development are significantly more likely to indicate at the baseline

364

that the main issues that they face are related to product development

365

or to the identification of the product market fit and really to the market validation.

366

So they're really facing higher level questions which concern the nature of the business model and the value proposition.

367

On the other hand, firms at high levels of business development were significantly more likely at the baseline to indicate

368

that the main issues that they were facing were relating to expanding the business and customer base.

369

So this is very much in line with our interpretation of the findings

370

that the scientific methods helped entrepreneurs at a lower stage of development to address questions

371

that were probably broader and that concerned the value proposition.

372

But in line with this, the outcome of that application, the method didn't necessarily lead them to obtain immediately revenue,

373

but probably to improve their business model,

374

to grow their, like the strength of that business model as represented by an increase in the number of employees.

375

Similarly, we are also conducting some text analysis regarding the way in which these entrepreneurs talked about their value proposition at the baseline.

376

And what we are seeing is

377

that entrepreneurs at a lower degree of business development were significantly

378

more likely at the baseline to describe their value proposition in a way that expressed a higher degree of uncertainty and hesitation.

379

We also try to provide evidence of the mechanisms by looking at what these entrepreneurs do after the training.

380

In particular, we look at the extent to which they make changes to their value proposition.

381

To do so, we use the business model canvas as a way of representing the entrepreneurial business model.

382

And I'm sure that this audience is quite familiar with the business model canvas

383

and the fact that it's composed by different elements, for example, the value proposition, customer segment, customer relationships, and so on.

384

So we look at whether entrepreneurs made any changes during the observation window to any elements of the business model canvas.

385

And we also look at the extent to which these changes were radical versus incremental.

386

And consistently with our theory and our interpretation of the results, what we see is

387

that firms at a high levels of business development are significantly more likely to engage in incremental versus radical pivot.

388

Once again, in line with the idea

389

that entrepreneurs at a low stage of development use the method to go back to the drawing board

390

and reconsider the way they're doing business.

391

but this does not necessarily lead them immediately to observe revenue.

392

On the other hand, firms that are at a more advanced stage of development have a more established value proposition, so use the method to improve it, to fine tune it.

393

And so they're already making revenue and this leads them to essentially make even more revenue.

394

We also try to exclude alternative explanations. For example, one that we consider was that maybe these results

395

that was associated with a higher

396

or lower degree of this development was perhaps driven by the experience of entrepreneurs in the industry

397

or experiencing running startups or managerial experience.

398

But none of these explanations was related to our results.

399

None of these alternative explanations essentially held.

400

Alternatively, we thought maybe this is is confidence that is really driving these results.

401

Maybe the more developed firms are more confident and so maybe this is what is actually driving our results.

402

But once again, we also ruled out this alternative explanation.

403

So to conclude, what we find is that overall

404

treated firms grow more than control firms and

405

if we look at the distinction

406

when it comes to performance as measured in terms of revenue

407

if we look at the distinction between less developed

408

and more developed businesses what we see is

409

that less developed business actually for them the treatment has a negative effect on performance

410

but it has a very positive effect on performance for more developed businesses now we believe

411

that these results show um contribute to existing literature on decision making

412

and strategy by showing an important boundary conditions for the application of the scientific decision making approach

413

and more in general about systematic approaches to decision making

414

and in particular it also highlights the complementarity between between the development of a theory about the firm and experimentation,

415

which is in contrast with the idea that entrepreneurs can really delay important choices and experiment at any stage of their journey.

416

We also believe that these results have important practical implications and they provide actionable tools for entrepreneurs.

417

We are in the process of developing some case studies as well and tools for entrepreneurs.

418

And we also believe that there's an important practical implication

419

that is particularly dear to us as educators in addition to being scholars,

420

which is basically that these results show that it's not just the content

421

that we teach in the classroom that makes the difference, but it's also the way in which the content,

422

it's also about teaching students or entrepreneurs how to use that content.

423

Because if we think about it, what we delivered in our control classroom was very similar to what we deliver in any management course.

424

But the difference in the treatment in this case was really about teaching entrepreneurs not only the tools, but also how to use them to develop a theory,

425

how to use them to test the theory.

426

So we believe this is something that is relevant for us as educators.

427

So what are the next steps?

428

We have presented today this paper, but this paper is really a part of a larger project

429

titled Scientific Approach to Innovation Management that we are conducting together with Alfonso Dambardella, Arnaud Camufo and other colleagues at Bucconi,

430

at the University of Maryland, at Erasmus, at the University of Lisandes, at the India School of Business, Coen University and Shanghai University.

431

More in general what we are trying to do with this project is

432

that of understanding and answering a broader question

433

which is what type of decision -making processes can help firms navigate uncertainty

434

and we hope with this project to be able to explore the antecedents associated with these decision making processes

435

and the implications

436

so we are in the process of setting up multiple randomized

437

control trials in different countries italy the netherlands uk colombia brazil china india

438

and tanzania and we hope

439

that this will give us the opportunity to further validate the robustness of the results

440

but also to explore other nuances and boundary conditions to improve our knowledge.

441

about decision making in the context of uncertainty.

442

So I would like you to thank you very much

443

and I will pause here and look forward to any feedback and suggestions.

444

Thank you.

445

So Elena, we actually have one question in the chat

446

so maybe we can start with that one and in the meantime

447

if people think about any other questions we can take them um

448

so since we are live online ziyan would you like to unmute yourself and ask the question

449

oh yeah can you hear me yes we can all right yeah thanks elena

450

and the care for the wonderful research is very interesting

451

and rigorous so i just have a quick question regarding the h2 uh especially in terms of the theory

452

so i think the research argument now centers on the benefits of scientific approach

453

but it also has the cost side of that approach for

454

so for the higher development of business it also means

455

that it's more costlier for them to do to adopt

456

that approach because they have larger sum costs if they use that approach.

457

So I wonder if

458

that also means the higher development also means they are unlikely to employ a scientific approach in reality

459

and that will also lead to the opposite prediction.

460

So it's about whether they use it or not rather than the approach is beneficial or not.

461

Thank you for this question.

462

We also look at costs actually in some supplementary analysis

463

and what we see is

464

that indeed the scientific approach leads more developed firms to basically higher expenses

465

so it is more expensive like they spend more

466

when they use the scientific approach so this is in line with with the prediction

467

that the scientific approach leads to higher costs not necessarily higher sunk costs but certainly higher costs

468

We also look at whether the value added, so whether, essentially, because we see an increase in revenue, and we see an increase in cost,

469

so a follow -up question could be, is it possible that increasing revenue is simply driven by the increasing cost, and so the value is simply transferred to the customer,

470

there's no value retention for the firm, and what we see is that we see an effect on value added.

471

So the scientific method leads more developed firms to also generate more value, which we believe is quite interesting.

472

I hope that answers your question.

473

Let me also add something about, right, because I think Elena was writing in talking a lot about empirics, but this is also a conceptual point, right?

474

And so the literature that looks at business development is really emphasizing how the level of development is not related to, for instance, age or cost.

475

So, of course, I think the example

476

which we showed might have primed you in that way because that shows two extremes on the continuum.

477

But usually the level of development is really about what are the problems that these firms are facing.

478

And so costs, and especially some costs which are irreversible, might not really be central in that particular case.

479

But I think it's interesting what you're pointing out about sort of what is the cost of using a scientific approach,

480

right? so i think this is also a very interesting question for for future research i see

481

that we also have a we have a question in the chat

482

and we have a hand raised and i believe

483

that the hand was raised first so why don't we hear from alexander

484

we can all hear you Alexander

485

oh it was a mistake okay

486

so there is another question in the chat from Alfonso Alfonso

487

would like to mute yourself okay just very simply uh i

488

would be curious to know how do you rule out

489

that higher developed firms are not just more skilled or experienced to adopt the approach

490

okay thank you for the question so um what we what we look at is um whether the experience of

491

more developed firms i mean obviously more developed firms are also tend to be uh more experienced

492

and we we basically uh try to see whether the effect

493

that we observe is is explained by experience whether experience leads to the same outcome

494

and we don't see So we don't find that more experienced firms

495

have a positive and more positive um like result when they use the approach

496

on the other hand when we look at the level of development we see the effect

497

does that answer your question yes thank you

498

I see Addis being spotlighted.

499

Addis, do you have a question?

500

Yes.

501

I actually left a question on the Q &A part.

502

There's a chat, so I think it was not clear.

503

It was an interesting presentation.

504

Do you hear me?

505

Can you hear me?

506

Yes.

507

Hello.

508

Hello.

509

Good to see you and it is an excellent project

510

so I have just one simple clarification question so

511

when you go on the post hoc analysis identifying the mechanisms you mentioned a few mechanisms

512

that kind of distinguish less developed and more developed businesses

513

and it felt to me more like they are the characteristics of the business themselves than mechanisms

514

so I was kind of yeah can you say a little more, as you say, in less developed business ideas, you know, owners were not very sure,

515

they had certain uncertainty, and I felt that was the very definition of less developed business idea.

516

So how do you kind of differentiate between the nature of the business idea itself, so it's not developed, versus the mechanism by which,

517

you know, the treatment helps them to, you know, to perform better? so just to check

518

that we understood the question um we understood the premise can

519

you maybe elaborate a bit uh more on the the question please

520

so yeah my question is are the characteristics the characteristics of the business idea

521

so highly developed versus less developed business idea

522

different from the mechanism you say as you know drivers for the performance outcome you observe

523

So I kind of see a mix between, you know, the characteristic of the business ideas and the mechanisms that connect the,

524

you know, the business ideas to the success, the performance.

525

Okay, so let me elaborate a bit on that, on the answer to this question and Chiara maybe can elaborate

526

and then you will tell us whether it answers or not your question.

527

So essentially our idea is that firms at a different level of business development face different problems.

528

So firms at a lower degree of business development are still validating the value proposition.

529

And this is the first point that you were suggesting.

530

And firms at a later stage are fine -tuning the value proposition.

531

So your question is, we understand that these are the two characteristics.

532

Tell us more about the mechanism.

533

And the idea for us is that for firms that are still developing the value proposition,

534

the method helps them make some higher level decisions, which can be, you know,

535

what is the customer, the target customer that we want to face.

536

So obviously, the method at this stage is relevant.

537

It helps them decide.

538

But this is like one out of several important choices that they're making.

539

So this might lead them to make more investments in a certain customer segment.

540

It might lead them to hire someone who is an expert in that customer segment,

541

but it might not necessarily translate into revenue immediately.

542

But the other thing that we are also thinking about in terms of the mechanism is

543

that this is like a search process in like a broader space, because many there are still many degrees of freedom.

544

So the entrepreneur has to go to I mean, the scientific method works in an iterative way.

545

So obviously each divided theory is further developed over time through these iterations.

546

And so when there are still many high degrees of freedom, first of all, it takes more time to cover the search space.

547

But also each test can be in a way less informative because, or let's put it this way,

548

might be more difficult to run all the like to develop the theory

549

and run all the different tests because if you think about the number of combinations that this entails is actually quite high.

550

On the other hand, if you compare the opposite situation, high degree of business development, very fine -tuned preposition, the search space is much smaller.

551

So the method, in terms of mechanism, to answer your question about the mechanism, can be applied in a much more focused way.

552

And so it quickly serves in providing answers that are like the missing pieces of the puzzle.

553

And then there is a complete picture,

554

which is the finalized fine tuned value proposition which can directly generate revenue.

555

I don't know if Chiara wants to add anything to this.

556

Sure, I think what I'd like to add is

557

that I think Addis went straight to the most difficult part of any field experiment

558

which is you know showing what the mechanism is

559

and obviously as this presentation shows these are preliminary results

560

so this is really the part of this project where ellen

561

and i are spending more time now

562

so really trying to show more clearly what is the journey

563

that these firms are going through

564

so i think as you mentioned the idea with the problems is not not

565

so much to show that they are the mechanism

566

but rather to show that that's the starting condition

567

so these firms actually face different problems which is something

568

that is consistent with the literature but then the mechanism are these different changes

569

that they're making so we're trying to work more on pivot

570

and other intermediate outcomes to show that a little bit more clearly

571

so i think this question is very helpful for us to to really clarify

572

that we need to focus on that a little bit more

573

and i think at this point is also closely related to the question

574

that miriam has that i see in the chat

575

miriam would you like to unmute and ask your question okay now you can hear me right yes Yes,

576

my question is very much related to the previous one and

577

so the first part of my question actually is a comment

578

and it's to make sure that I understood basically the main conclusions of the paper

579

which is one size basically does not fit all.

580

So it depends on the stage of the business cycle of the specific firm, how this firm adopts the scientific method,

581

the cognitive method and it depends on the outcomes that you're looking at.

582

Now, my point here is whether it's about the way the different firms,

583

according to the stage of the business development, use the scientific method that is different,

584

or whether they make different decisions because they are in a different stage of the business cycle,

585

and therefore they have to make decisions that are different compared to firms

586

that are more like ahead in terms of the business development.

587

So if you want, my question is confused because it's not clear to me what somehow what I want to ask,

588

but so whether this is an issue concerning the way firms use the scientific method,

589

whether it's a matter of the decisions that different firms at the different stage have to make.

590

So what comes first here that explains your finding?

591

yeah i think it's a very clear question actually it's not it's not confused at all um

592

so maybe i can start and then uh elena please feel free to add

593

because it's quite a broad question anyway um

594

so i really like how you summarize this idea of one size doesn't fit all

595

which might be one of the key takeaways

596

and i think elaborating on this point what is interesting is

597

that especially for policy makers for accelerators for us as educators

598

this suggests there might be better ways to help these entrepreneurs

599

which are really early in their journey of developing a new business now i think the caveat also

600

that we need to keep in mind when we interpret these results is

601

that the study has a fairly limited duration right

602

so precisely for the reasons

603

that you're discussing about the different type of problems it might be

604

that these early stage entrepreneurs need more time to reach a

605

stage where we can really see the benefits of using a scientific approach

606

and so the fact that they're facing these different problems might imply that they need more time.

607

So hopefully this is something that we will be able to measure later on at some point, as Elena mentioned earlier, with additional data.

608

But I think this is an important caveat and something

609

that we're thinking about in terms of how to interpret the results in a fairly cautious way.

610

But Elena, I'd love to hear your thoughts about this.

611

What comes first, problems, decisions?

612

Absolutely.

613

Thank you, Miriam.

614

It's obviously a very important question, and I particularly think that from a practical standpoint,

615

your suggestion in the chart of looking at intermediate decisions,

616

because I mean, essentially what we are saying here is also that between growing the firm,

617

like defining the value proposition and making revenues and exploiting that value proposition, there's really a lot.

618

And so it would be really nice to be able to map.

619

the data generation process there with possibly looking at intermediate outcomes,

620

so this is in fact why we in the following studies,

621

these are things that hopefully we will have the opportunity to observe so that it will provide additional evidence in this direction.

622

With this specific study, we obviously are limited to what we measured, so we are trying to provide some evidence in that direction,

623

but it's certainly where I mean it's certainly the direction to go in

624

because it's a very important opportunity to map the decision making process

625

and understand more about how I mean about the process

626

that goes from making the decision to observe some intermediate outcome and then investing more until performance is observed.

627

Thank you.

628

Thank you.

629

I think we're coming very close to the end of the session.

630

So I'll leave it to Emily or Ryan to wrap up.

631

But what I want to say for sure is

632

that I would to thank everyone for being with us today and taking the time to attend our presentation

633

and we would really love to to hear any thoughts um any ideas

634

or suggestions so please feel free to reach out to us

635

and we would be delighted to continue this conversation thank you thank you everyone.

✨ Vídeo recomendado

60 frases

Vocabulário e dicas de fala para esta lição

Este vídeo tem 635 frases e 8602 palavras para praticar shadowing. A fala dura 56:26. O falante fala em ritmo natural, cerca de 152 palavras por minuto, próximo de uma conversa do dia a dia. 86% das palavras estão entre as 3.000 mais comuns do inglês; vale a pena estudar o restante antes de começar.

Vocabulário principal deste vídeo

15 palavras do vídeo que vale a pena aprender, com pronúncia e significado:

Palavra	Pronúncia	Significado
entrepreneur substantivo	/ˌɑn.tɹə.pɹəˈnɝ/	empreendedor, empreendedora
proposition substantivo	/ˌpɹɑpəˈzɪʃən/	proposição, proposta
mechanism substantivo	/ˈmɛk.ə.nɪ.zəm/	mecanismo
observe verbo	/əbˈzɜːv/	seguir, acompanhar
canvas substantivo	/ˈkæn.vəs/	lona
essentially advérbio	/ɪˈsɛnʃəli/	em essência, essencialmente
uncertainty substantivo	/ʌnˈsɜːtənti/	incerteza
hypothesis substantivo	/haɪˈpɒθɪsɪs/	hipótese
specification substantivo		especificação
tune substantivo	/tjuːn/	melodia
baseline substantivo	/ˈbeɪslaɪn/	referência
necessarily advérbio	/ˌnɛs.əˈsɛɹ.ə.li/	necessariamente
extent substantivo	/ɪkˈstɛnt/	extensão
experiment substantivo	/ɪkˈspɛɹ.ə.mənt/	experimento, experiência
versus	/ˈvɝsəs/	contra, vérsus

Phrasal verbs que você vai ouvir

Palavra	Pronúncia	Significado
look forward to verbo		ansiar por, estar ansioso para
rule out verbo		riscar
point out verbo		apontar
set up verbo	/ˌsɛt ˈʌp/	preparar
wrap up verbo		embrulhar, empacotar

Pronúncia para ficar de olho

O falante usa 55 contrações e formas reduzidas, como they're, we're, don't. Diga-as na forma curta, do jeito que você ouve.

Os sons de “th”: hypothesis /haɪˈpɒθɪsɪs/, gather /ˈɡæðɚ/
Os sons de “sh” e “zh”: proposition /ˌpɹɑpəˈzɪʃən/, essentially /ɪˈsɛnʃəli/, explanation /ˌɛkspləˈneɪʃən/, presentation /ˌpɹɛzn̩ˈteɪʃn̩/, beneficial /ˌbɛn.ɪˈfɪʃ.əl/
Palavras longas — acerte a sílaba tônica: entrepreneur /ˌɑn.tɹə.pɹəˈnɝ/, proposition /ˌpɹɑpəˈzɪʃən/, mechanism /ˈmɛk.ə.nɪ.zəm/, essentially /ɪˈsɛnʃəli/, uncertainty /ʌnˈsɜːtənti/

Como praticar com este vídeo

Ouça o vídeo inteiro uma vez sem falar e anote as palavras que você não conhece.
Comece na velocidade 0,75×, faça shadowing frase por frase e volte à velocidade normal quando ficar fácil.
Grave a sua voz e compare com o original, prestando atenção a palavras como entrepreneur, proposition, mechanism.

Gramática neste vídeo

As estruturas que o falante mais usa, com as palavras exatas do vídeo:

Estrutura	No vídeo
Present perfect have/has + particípio passado — uma ação passada que ainda importa agora	have been identified · has suggested · has identified
Voz passiva be + particípio passado — o foco está no que acontece, não em quem faz	are characterized · have been identified · could be met
Orações relativas who / which + oração — informação extra sobre uma pessoa ou coisa	one who starts · one who draws · someone who tests

O que é a Técnica de Shadowing?

Shadowing é uma técnica de aprendizado de idiomas com base científica, originalmente desenvolvida para o treinamento de intérpretes profissionais. O método é simples, mas poderoso: você ouve áudio em inglês nativo e repete imediatamente em voz alta — como uma sombra seguindo o falante com 1-2 segundos de atraso. Pesquisas mostram melhora significativa na precisão da pronúncia, entonação, ritmo, sons conectados, compreensão auditiva e fluência na fala.

Técnica de shadowing: leia o guia completo passo a passo →

Prática de Shadowing: Decision-Making and Entrepreneurial Performance: Evidence from a Field Experiment - Aprenda a falar inglês com vídeo

Quick Study: How Venture Capitalists Make Decisions

Anxiety is the Dizziness of Freedom｜英文Talk：焦虑是好事，恰恰因为现代社会的流动性强

Research Foundations: How to Turn Business Problems into Actionable Data

Commencement Speech 2012 - Full-Time MBA | SDA Bocconi

Game Theory #3: Rich Dad, Poor Dad

Trust the Process: How to Design Teams That Actually Work

Introduction to Economics Part 2 - Professor Ryan

GATHERING IN THE DIGITAL ERA

How to be a creative thinker | Carnegie Mellon University Po-Shen Loh

Talk Back: Why Arguing With Yourself Can Help You Flourish

Why you procrastinate even when it feels bad

Unit 1.1: Introduction to Business Management - IB Business Management

Vocabulário e dicas de fala para esta lição

Vocabulário principal deste vídeo

Phrasal verbs que você vai ouvir

Pronúncia para ficar de olho

Como praticar com este vídeo

Gramática neste vídeo

O que é a Técnica de Shadowing?