Experimental Physics and
Industrial Control System

Jeong Han Lee <[email protected]> · Sat, 21 Jul 2018 03:12:26 +0200

Hi Siniša,

 It was not only Michael, but unknown (hidden) persons. I think, I am

still useless and do-nothing one hidden person who supported him to do

this work.

 I was in the V4 core meeting a long ago without knowing anything,

still I am not doing anything useful for V4 and V7 at all. At that time,

I remembered the community had no money to develop the PVA gateway

service, which Michel had the draft proposal at that time.

 I asked to all, "I could fund, so what could I do?" I was told

"please talk to Bob".

 I walked through many internal reviews to get small money in my old

facility to support the EPICS V4 roadmap (gateway service) even if I

didn't (still don't understand yet) what we need for V4 services.

  Since then, Michael was starting to develop the PVA Gateway prototype.

 Maybe my decision ago made this conclusion today. I feel very sorry

for Michael. "If I didn't fund that money, what will be now." is my

thought while following up these emails.

 We are standing on everyone's shoulders, and we are developing

something new with and without conflicts. Nothing is new on Earth.

Indirect and directly all our works supports other codes to make our

IOCs running fine.

  HTH and Thanks,
  Han

On 07/20/2018 05:06 PM, Siniša Veseli wrote:

Michael,

Thanks for the response. I'll just add few more short comments, and then

I will not argue any longer.

1) I do very much appreciate your willingness to step up, maintain

various pieces of the base code, fix problems, etc. I have nothing but

outmost respect for both yours and Marty's contributions to EPICS

software and community. Things have gotten significantly better in terms

of performance on the C++ side over the last year or so, and I give full

credit to both you and Marty for this.

2) There will always be many external packages, modules, libraries, etc.

in existence that try to build on top of epics base. We can not and

should not control this. What we can control, however, are the

interfaces, APIs, and libraries in epics base that we present to users.

Ideally, regardless of whom I talk to, I would always get the same

answer on how to build a c++ client for a particular epics base release,

and hopefully this answer would not change very much every year or two.

3) Software will inevitably evolve with time and efforts to make it

better. There will always be differing opinions between group members

about which way to go. We will not always be able to achieve full

agreement between everyone, but as long as the core developers group

builds consensus around a particular issue, I would imagine that would

be good enough. I am not arguing that every little detail must be

discussed by some sort of committee, but things like adding new or

modifying high-level APIs, or differing opinions on a significant github

PR, or usage of PV request, those all probably warrant some sort of

proposal and a discussion at a face to face meeting.

Again, please do not take offense with any of the above. We all have a

common goal of making epics software better and easier to use, and I

simply hope that this email thread helps in a small way.

Best Regards,

Sinisa

On 7/20/18 3:04 AM, Michael Davidsaver wrote:

Sinisa,

On 07/19/2018 03:33 PM, Siniša Veseli wrote:

Hi,

Judging from Michael's comments, documentation, epics meeting slides,

etc. it looks like these new APIs are indeed meant to replace what we

had since v4.5 or so. While I am not necessarily opposed to such

change, I do have a few points I'd like to make:

1) EPICS in itself is probably not the easiest software to learn and

use. Having two sets of C++ APIs does not help and can be terribly

confusing. If I were new to Argonne, and someone told me to write an

EPICS C++ client for the first time, and then I were to discover that

in epics base we have client.h and pvaClient.h that seem to be

completely different, that we have two sets of examples, different

documentation, and in some case different behavior between the

clients, I would ask myself "What is going on here, and who the heck

designed all of this...".

Can you name even one part of the EPICS ecosystem which hasn't had

multiple implementations, APIs, and/or wrappers?

How many driver frameworks are there?
How many libca wrappers?
How many outright independent re-implementation of CA?
How many OPI systems?
How many archive engines?
How many alarm managers?

I think you're trying to hold me to an ideal which has never existed

in this community.

Mostly I think this is a good thing.  EPICS is a toolkit, not a

product line.

Of course anything can be taken to excess, but I think I would have to

go quite a bit

further before I could be seriously accused of this.

And just to pick a nit.  Your count is one short.  The ChannelProvider

API is just as

much as much a supported public API as either pvaClient or my new

entrant.  There is

already quite a lot of confusion and a decades worth of outdated

examples floating around.

2) Maintaining and supporting two sets of high-level APIs is not

feasible in the long run. Resources that we have for epics

development are scarce, and those should be used wisely. This is

something that anyone sitting on the council should be concerned

with. Why would we have Marty or anyone else spend lots of his time

on pvaClient and pvDatabase if we are going to get rid of it in 6

months or in a year? I would personally rather spend my time on

something that will have a lasting impact on the project, rather than

fixing a bug or two in something that is viewed as temporary.

You want permanent software?  I'm not sure I can help you with this :)

I too would rather be spending time on other things.  Squashing bugs

in PVA,

or working on a driver for a neat new piece of hardware.  However,

I've found myself

spending more time fighting awkward and error prone APIs than doing

productive

work.  I've been into the guts of this code base, and a few similar

ones, so I know

what it's really doing, and I know it doesn't have to be this hard!

Others have reach the same conclusion.  However, instead of fighting

to improve the

situation, they mostly written PVA off and moved on.  imo. this is

part of the reason

this "new" code is already a decade old without seeing widespread

use.  This isn't how

I work.  I'm willing to work to incrementally improve existing code.

Like EPICS Base,

and now the PVA modules.  It isn't glamorous.  It doesn't publish

papers.  And when

it goes well, no one notices.

Rather than questioning why I've written some (rather thin) wrappers

to try to

rationalize PVA, I wish you would ask me why I haven't just tried to

rewrite

the whole thing?  Or why I haven't gone off and started again from

scratch?

3) I would hope that EPICS development project has some sort of

coherent strategy in terms of what gets developed and why, what are

the recommended user interfaces, etc. Module owners have a wide

latitude and control in terms of how they want things to be done.

However, in a case where someone feels that a new high-level C++ API

is needed because the old one is just not good, or if they wanted to

make API changes that affect users or other downstream modules, I

imagine there would be some group discussions and planning around

that, so that we all know what is going on. We may not all agree

about particular set of changes, but at least everyone would get

their say and the group would build some sort of consensus around the

direction that the project is taking.

It seems like there was not much discussion in this particular case.

New interfaces appeared in the base code at some point, and are

finding their way slowly through the rest of the system, so now we

have two sets of APIs, and that's just how it is... Depending on who

you talk to first, you might go this way or the other for developing

your own code, and hope that the path you've chosen will not be

deprecated in the next release. This just does not seem to be a good

thing as far as EPICS users are concerned.

Ok, you have me here.  I didn't solicit input from a certain

committee.  Several

of the individuals, yes.  But I only presented the results.  I had

reasons for

avoiding this.  Some I'm not so proud of.  So here is your reward for

reading this

far.  Some dirty laundry.  It isn't a secret that I've totally failed

to figure out

how to work with Marty.  That relationship when wrong the first time

we met, and has

continued to go wrong for 8 years.  I've been unable to find a way to

approach

him about changes to his code without his getting defensive, and the

discussion

degenerating into an unproductive stalemate.  This has been repeated

many times

over the years, and I'm tired of it.  So rather than pick yet another

fight with

Marty, I made something which works for me.  From my prospective it's

already

paid off in time saved.  I think it will be useful for others, which

is why I

documented it, but only time and bug reports will tell.

Still, I would have you note that at this point, I've spent the

majority of my

career working with Marty's code.  In Base, asyn, and PVA.  Some I

like, and

some I don't.  There are certainly aspects I wish were done

differently, but

on balance these are good platforms to work from.  This is why I was

willing

to step up as maintainer, first for Base and now for PVA.  I just wish

Marty

could see past my criticism of PVA, and consider what it means that I

care

enough to try to change it.

I am not arguing to maintain some sort of status quo here, but I am

not sure I completely agree with how are changes being introduced at

the moment. If new client/server APIs are going to be significantly

better, if we need to change the way we use PV requests in our code,

etc., that would all be fine with me, as long as this was discussed,

consequences were understood, transition plan for the old code was

developed, etc.

This email is not meant to offend anyone, if I did so, then I am

sorry and I apologize in advance.

I do appreciate Michael's clarifications on what happened and how

things are at the moment.

Sinisa

On 7/19/18 1:42 PM, Marty Kraimer wrote:

On 07/18/2018 08:05 PM, Michael Davidsaver wrote:

On 07/18/2018 01:20 PM, Siniša Veseli wrote:

Hi,

I realize I have not been attending the working group meetings

lately, so I apologize in advance if I missed something.

Reading through some of the github issues that seem to be getting

a lot of attention recently, it seems like we are getting a

completely new set of PVA client/server C++ APIs, which raises

several questions in my mind:

1) Are those APIs becoming the "official" EPICS PVA client/server

APIs? In other words, if I am starting a new C++ project based on

EPICS7, should I start with the new interfaces, or work with the

ones available in the most recent 7.0.1.1 release?

My new client API (pva/client.h) is actually in the 7.0.1.1

release.  I wrote this last summer.  It is the result of my

experiences using the ChannelProvider API with the PVA gateway

prototype, the P4P client,

the CLI tools, and in unittest code for QSRV and the PVA gateway.

pva/client.h is my attempt to distill out the code common to all of

these use cases.

Since 7.0.1.1 I've successfully used it in the new PVA link

support, some of the CLI tools, and recently

rewritten the P4P client around it.  In doing this I didn't have to

make any incompatible changes, just to

add a few missing pieces and make bug fixes.  So from this point I

expect it will remain stable.  I plan to

apply it to the PVA gateway the next time I iterate.

As for the (very new) server API.  This does not attempt to be

fully general purpose, eg. it won't be

enough to build a gateway.  Instead, it is a direct decedent of the

server code common to QSRV and the unittests

of the PVA gateway prototype, which is will replace.  This common

code is what has held these two otherwise

unrelated projects in the same source repository.

So far it's only been use behind the P4P server API.  I'll consider

it stable once it has been successfully

used a few more times (by myself, or others).  So I suppose you

could call it a "technology preview" in the

upcoming 7.0.2 release (a step I think should be mandatory for all

new APIs).

As for "official"-ness; these APIs will be released, and I will

continue to use them in my own projects,

and as a tool to test the underlying PVA client and server code

(hopefully finding bugs before you do).

Should you use them?

I'd like to get feedback from people like yourself who also have

experience with both the low-level

ChannelProvider API, as well as Marty's

pvaClientCPP/pvDatabaseCPP.  My goals are that these new APIs

should be easier to use,

harder to break, and perform just as well.  If I'm not meeting

these goals, then I'll do something about it.

2) If #1 is true, what becomes of the current pvaClientCPP and

pvDatabaseCPP APIs? Will those be deprecated? Or are we going to

have two separate sets of high-level C++ APIs?

As you allude to in #3, Marty and I disagree about pvaClientCPP and

pvDatabaseCPP, of which I am not a fan.

I don't know what his plans are, but I have not, and will not,

dissuade him from continuing to develop these modules.

So yes, it looks like there will be two sets.

3) It looks like there are some rather significant differences in

behavior between the new and existing interfaces. Have those

differences been summarized anywhere?

No.  I can only refer you to my documentation.  See "Client API"

and "Server API", and particularly the "API usage Examples".

http://epics-base.github.io/pvAccessCPP/

I would imagine that making API changes like this, and for a

project that numerous sites depend on, requires some sort of

process: somebody comes up with a proposal and plan, there is a

discussion about advantages/disadvantages and about the

consequences for the existing code and for the future development,

and (most importantly) there is an agreement between the

developers as far as the direction that project will be taking. If

this was discussed in one of the v4 meetings already, sorry that I

missed it, and I would appreciate if someone points me to the

document where all of this has been written down.

My motivation for this tool smithing is that I found the existing

options quite awkward, error prone,

and unlikely to change.  I know that others have similar

experiences.  I developed some patterns and

utilities to cope with this.  I also dislike copy+pasting code

between projects.  These new APIs are

the end result of this process.  I know they are useful to me. I

think they will be useful for others,

but only time will tell.

On a personal note, few years back, after we all decided that the

new client API (pvaClientCPP) will be implemented, I had to put in

a fair amount of work to replace my own high-level API with the

official one, and I would not be exactly looking forward to yet

another iteration of the same. However, if this was already

discussed and agreed upon, then so be it.

If you're happy with pvaClientCPP, then I see no immediate reason

for you to change.  The low level

ChannelProvider API on which all of these high level APIs are based

will be maintained (warts and all)

for the foreseeable future.

Eventually, I hope that you will conclude that these new APIs would

make your job easier.  But only you

can be the judge of this.

Michael is creating a new client  API for pvAccess.

https://github.com/epics-base/pvAccessCPP/blob/master/src/client/pva/client.h

He has used this API as the basis for
1) pvTools. In particular pvget and pvput
2) pva2pva. Both qsrv and gateway
3) p4p

The issues

https://github.com/epics-base/pvAccessCPP/issues/116

and

https://github.com/epics-base/pva2pva/issues/21

Show that pvTools and qsrv have broken the exceptions of existing

clients and servers.

In order to support both the existing clients and Michael's new

client API then there will have to be two implementation of at least

pvTools and qsrv. If we continue with two client API's there will

probably be a need for more dual implementations.

Marty

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System