[Kommander-devel] I hate Kommander

[Eric Laffoon]

On Tuesday 20 September 2005 2:26 am, Andras Mantia wrote:
> Hi Eric,
>
>  Ok, I'm trying to be calm today. ;-) Anyway, the weather can easily
> cool me down as it is 11C outside and less than 20 inside. It seems we
> have to make fire in the stove.
>
> On Monday 19 September 2005 23:33, Eric Laffoon wrote:
> > You're cracking me up. Kommander is a paradox that you should learn
> > to see.
> > 1) It is "different thinking" than an experienced developer.
> > 2) Ironically it is easier for non developers to pick up.
> > 3) You are an extremely accomplished C++ developer.
>
> So it is better if I remain at C++? ;-)

Yes. Let's wait till KDE5 to write Quanta in Kommander. ;-)
>
> > Kommander. The operational term is learning the particulars. Once you
> > understand them in Kommander it is assuredly faster because you don't
> > have to do anything but logic and operations.
>
> That would be the ideal way. My opninion about Kommander is that it
> should not stay in you way. It should be no more than a tool which
> provides you widgets and ways to modify/access those widgets and gives
> some structures to the code, by making it possible to associate code
> with widget actions.

So far I agree completely.

> It should allow me to write whatever I want in the 
> language I choose, be it bash, perl, ruby or what.

Unfortunately this has proven not to be completely realistic. If you're 
willing to accept certain limitations or do this in a more difficult way then 
yes. Conversely if you're willing to create extentions for each of these 
languages for Kommander this could be achieved. However this is both a lot of 
work and effectively reduces the ability to maintain more complex projects in 
the future that use multiple languages. Personally I would support it only 
behind a number of other improvements.

> I choose bash, as I 
> don't know the others and most people at least understand it somewhat.

bash is certainly common, but I don't think most people understand it a 
little, unless you mean very little. Our previous favoring of bash from 
Marc's work had some key differences. For one it was not nearly as powerful 
as Kommander is now, so it really cannot be compared. Another factor for us 
is how many support requests we might expect to get from people trying to do 
things. Unfortunately in this regard bash is the most likely to create 
problems and the least likely to be easy to integrate. Finally there is the 
rather serious issue that you can watch the widgets draw if you have even 
just a few calls because it's so terribly slow.

I'm not against bash support, but if you want to do it all with bash then you 
are staking a place with the dinosaurs. We need to improve our current 
facilities and documentation, and at least the facilities are not that bad. 

The point is... every user is going to come to a tool having to gather some 
understanding of it's tradeoffs and having to learn something. With Kommander 
we have tried to keep the bar very low, but you're asking for the bar to be 
floor level for a very difficult and quirky language where you want to do 
comparatively intermediate to advanced things with it. That's incongruent and 
it's also not recognizing the inherent challenge. Suppose I want to use Rexx? 
With your analogy I need to write a DCOP library for Rexx or shell out to 
bash which is hundreds of times slower. I happen to like Rex from my OS/2 
days. So the real question is how to manage what has to happen, which Michal 
and I have agonized over for some time.

> The main problem that arised was that Kommander specials are evaluated
> before the code is passed to the script interpreter. This is what
> causes the madness about not being able to pass variables between your
> script language and Kommander. Some time back I already proposed a
> solution, but it was ignored. It was about not doing in this way. Take
> this example:
> a="first second third"
> for i in $a
> do
>   @List.insertItem($i, -1)
> done
>
> This does not work with the current Kommander, because @List.insertItem
> is evaluated before the for loop even starts. Instead of this right now
> you have to use something like:
> a="first second third"
> for i in $a
> do
>   dcop @dcopid KommanderIf insertItem List $i -1
> done
>
> It is ugly (but maybe @dcop works as well), but the problem is that it
> is slower. I agree, but Kommander is not for time critical
> applications, so this kind of slowness should be acceptable, especially
> if there is another way to do it (the "Kommander" way).

Yeah, here's the thing... 
1) It's not just slow, it's hideously slow. I mean watch the widgets draw and 
set when used with start up scripts. 
2) What you have there can generally be done in one line as opposed to the 5 
lines you have and the operation there is much faster too. This is 
particularly noticable when filling a tree or listbox with hundreds of items 
where there are orders of magnitude difference in speed.
3) While we're on the subject, bash is ugly, inconsistent and an invitation to 
get questions how to do things.
4) By providing a point and click interface for something that only takes a 
line or two of code the only person who is going to have a problem with it is 
a professional coder who doesn't have to look at docs for every line and 
wants to just type what he knows.
5) The most logical way to support common languages is to use a common 
framework for execution logic and then plug in operation scripts in various 
languages. Say we have python, perl and ruby programmers working together and 
some perl work needs to be upgraded by the python coder. If it's just a small 
script he could rewrite it, but if it's the whole dialog the benefits of 
modularity are lost.

I would remind you that the speed difference is around 100 times faster for 
internal calls. In my installer application setting four checkboxes with 
external DCOP caused obvious flicker as you saw them set. They come up clean 
with internal DCOP. If we discard all the internal DCOP work we can forget 
about extending Kommander for Mainwindow use. But in fact the old external 
DCOP is still supported. All that should be required for people who want to 
watch their application dribble onto the screen so they can be cozier with 
bash than point and click is to develop tools to support it. ;-) As far as 
I'm concerned they should be in Kommander. My DCOP Aide actually probably 
does just what you need.
>
> My suggestion was that for every language Kommander supports we provide
> the code that will be put instead of @List.insertItem or whatever. So
> the user still works the first version, and as Kommander knows this is
> bash, it will replace
> @List.insertItem($i, -1)
> with
> dcop DCOP-ID-OF-THE-EXECUTABLE KommanderIf insertItem List $i -1
>
> Same for other Kommander specials, like @Message.

That's hideous! For a second I thought you had a flash of genius. What about 
converting the other language calls to Kommander calls with a pre-processor? 
Doing what you're suggesting with bash is a recipe for disaster when we 
enable Mainwindow with menus and actions. A small application would be a 
total dog and people would be sending us bash questions because for whatever 
reason that's the language they had to maintain or somebody steered them 
there. For the record bash is hideous. It's useful for some things but could 
easily be replaced by perl or python today.
>
> In some languages there might be direct DCOP bindings, so you will not
> have to use the "dcop" command line tool to communicate with Kommander
> and will be much faster.
> The ways to access DCOP methods in a language will be described in
> external files, like in case of Quanta's DTEPs. Two methods should be
> offered for each language:
> - a way to call DCOP methods directly (if there is such way)
> - a way to execute external applications. This will be used to launch
> the command-line DCOP tool if the first methods is not available

This is certainly something I'd like to support. Who's going to do it? My 
thinking is that if we can get the other things I want done we will find 
people wanting to work on this. I think many languages already have DCOP 
bindings. This is a good example of a tool that creates external DCOP.
http://kde-apps.org/content/show.php?content=13125
Note it's made with Kommander.

What needs to be done is the fundamental improvements to Kommander...
1) Mainwindow and Wizard fixed
2) Actions instead of scripts
3) Parameters for signals/slots
4) New editor with highlighting and auto completion
5) Project tools/common install and run patterns and tools
6) plugin and tools framework
7) Better docs and enhanced site support including KStuff and XML pages 
hosting projects

With these things done and thorough support for using Kommander to extend 
Kommander it would be easy for scripters to take frameworks (like we did with 
DTEPs) and create enhanced support for languages and features. What we need 
is to empower Kommander users to become Kommander developers. The concept 
couldn't be any simpler. We tried it with Quanta and it gets a small result 
but it's more applicable to Kommander.
>
> Clear, simple and will just work in a way the coder expects.
> This way you can explore your language's full possibilities without
> problems and still communicate with Kommander.
>
> Now, if you want to go the Kommander way and use @for, @if, @String and
> whatever, you declare at first that this is a Kommander script, and
> than you don't write code in other scripts, except inside @exec or
> maybe between @execBegin and @execEnd. Still in this case a fundamental
> problem must be solved: to pass variables between the Kommander part
> and the other script part, most probably with the above solution. How?
> For example:
> #!Kommander area
> ... some code with Kommander specials
> @execBegin(#!/bin/sh)
> ... some bash code
> @execEnd
> ... some other Kommander code

I don't think that will work like you think because eventually you want to 
pass a Kommander value to a bash variable in a loop. The bottom line is that 
it's just going to be too unreliable when you do this. Besides that's really 
ugly. If you can write if/elseif/then/endif and call Actions and easily 
navigate that it's better.
>
> This could be replaces by:
> dcop @dcopId KommanderIf execute Areaname
> ... that some bash code
> dcop @dcopId KommanderIf execute Area2name
>
> So to the Kommander areas names are assigned (automatically) and the
> DCOP method executes them. It may be that the execution is not a simple
> DCOP method, but something like
>  kommandervars=`dcop @dcopId KommanderIf execute Areaname`
> so you get back all the Kommander variables in an array.

You missed a lot of what we went through. Aside from bash being ugly and slow, 
did I mention bash is ugly and slow, we encountered problems in logic and 
particularly loops that could not be adequately managed. I don't recall the 
particulars and it's 3 am now... In any event...
1) Users are going to need to use DCOP calls or functions to access widget 
values so this is a given in any language.
2) Structural flow with Kommander is little more than @if/@endif, 
@switch/@case/@end and @for/@endfor. The new script language simply adds 
else, elseif and nesting while not including the switch statement. That's not 
difficult to do with a function browser and some people may even be able to 
memorize this given enough time. ;-)
3) The new script language is very fast and can manage local variables.

The only thing missing is passing parameters with signals/slots and possibly 
with calls to Actions/Scripts.
>
> Or if this sounds too complicated, remain at the current solution, that
> the Kommander part starts to run, and when at exec you execute that
> part in the interpreter. Still, whenever in that part there is a
> reference to a Kommander special, replace it by a dcop call.
> So
>  bashvariable=@KommanderVariable

This still sucks! I wish you'd take my word for it. Let's say that in the 
above line your @KommanderVariable is set after the scipt initiated and was 
not certifiably static. That means that when the KT is interpreted it will 
parse all the @ values and then proceed to parse the bash. The problem is 
that if you set @KommanderVariable above with bash and then run the script 
you will set $bashvariable to an uninitialized or incorrect value. This is 
too easy to do and it means we will answer a lot of emails. 

The fundamental problem is that you have two asynchronous executions going on 
and you have to make sure there is no precedence issues. While you can make 
DCOP calls from bash it's not really DCOP enabled. In PHP the line above 
would become...
$phpvar = exec(dcop @dcopid KommanderIf getVariable KommanderVariable);
>
> would become
>  bashvariable=`dcop @dcopid KommanderIf getVariable KommanderVariable`
>
> Same in the other way
>  @KommanderVar=$bashvar
> is replaced by
>  dcop @dcopid KommanderIf setVariable KommanderVar
> just before execution.

Andras I'm really agonizing here. I need to find out where Michal is and get 
the database plugin released. As for what you're suggesting I'll make it 
simple. NO! We already have this functionality in the Kommander script which 
is fundamentally the same as the @ commands with the addition of variables 
and some syntax changes which are designed to be very consistent and simple 
with an object oriented context. It's quite similar to Javascript. The point 
is we have a tool to develop statements where you're not familiar by point 
and click and that means even a novice can use it. With a good editor we can 
do it with auto complete. It's as close to a language-less language as 
possible and is primarily for access of widgets and logic flow. From there it 
can easily call Actions/scripts to execute processes in any language.
>
> Bigger problem is to access the bash variables *after* execEnd. Two
> solutions:
> - don't do anything, require to save the variables in the bash part to
> Kommander variables
> - automatically save them in @execEnd using some predefined names like
> appending "_bash" or "_perl" suffixes.
>
> > Houston we have a problem... My Samsung Digimax 301 is not on the
> > list for gphoto2 and is not recognized even when I provide the device
> > with --path=. My camera is however USB mass storage so fortunately I
> > don't need gphoto2.
>
> Don't worry, I can give you an example without gphoto2. Just replace the
> files=`gphoto2 ...`
> line with
> files=`ls -1`
>
> If the echo is at the end, the echo in the Delete script works,
> otherwise it doesn't. They are completely independent scripts!

Then we need to look at what inconsistencies we have and fix them or Michal 
can address this and explain what is happening.
>
> > >  - press "Delete". It should list the selected items. Problems:
> > >  1) echo "selection=$selection" : does not work
> > >  2) @exec(echo "selection=$selection") : does not work
> > >  3) @debug($selection) : outputs "$selection"
> > >  4) @Message.info($selection, selection): outputs "$selection"
> > >    Conclusion: there is no way to print out the $selection
> > > variable.
> >
> > There's a lot of problems with this because you're still mixing bash.
>
> What is the problem here? I was told that by default the scripts are run
> in bash.

In discussions with Michal we have different things going on. I'd need to look 
through a lot of notes or he could chime in. Effectively for instance 
population text is different than widget text, however I seem to recall he 
expressed differences between buttons and scripts which did an action and 
widgets which hold a value. I'm not sure exactly what the deal is with this 
now as I am not sure exactly why this distinction exists, but it does affect 
things.

> Thus 1) should work. 2) should work even more. I simply 
> execute an external command. It  is not about "echo". @exec(ls) doesn't
> work either. Even more funny:
> selection=`dcop @dcopid KommanderIf selection FilesList`
> echo "selection=$selection"
> @exec(ls)
> @debug($selection)
> @Message.info($selection, selection)
>
> Do you know what happens if I execute this code? "Kuanta/KDevQuanta" is
> executed!!! Now should I laugh or cry? ;-) There is no sign of
> "kuanta" (the name of the executable) there....

That is indeed curious.
>
> > It would be better to switch on Kommander script.
>
> I don't want to. I want to do in my own script language! This is the
> point.

Maybe you should write a bash fork? Believe me when I tell you that the 
decision to proceed in this way was based on a fair amount of banging our 
head against the wall with the objectives you stated. They don't work... 
unless of course we're pretty well done with improving Kommander.
>
> > > - now add an
> > > echo ""
> > > line at the end of the Fetch kommander text. Surprise, suddenly the
> > > 1) works! 2) still doesn't work, 3) and 4) prints again literraly
> > > "$selection".
> > > And this is not only about echo, as basicly no bash code works in
> > > this part.
> >
> > Well first of all I'm not sure exactly what you're expecting.
>
> I would expect that all for items above prints out the selected items
> from the FilesList. Oh, first, I would expect that the code I write is
> executed at all...

Clearly this is how the widget is processing stdout. I recall Michal 
expressing his reasons for this but I don't remember what they were. 
The point is that the code your wrote does not work in a widget but does work 
in a button or script. So instead of saying it doesn't work I think you 
should either think about moving it or we should review the code and look at 
why this is. Of course info from Michal would help.
>
> > Each
> > Kommander Text is local so in order to work between them you need
> > either a Kommander global or use environment variables with bash,
> > which Kommander supports with @env IIRC.
>
> Using globals are ugly, but I accept that locals are not available.
> Using enivornment variables is a good idea, I did not think about them.
> But I think my solution at the beginning is much better.

I think you should plug in much more than just your current needs and rethink 
things. A number of critical decisions are forced on us when we move into 
this paradigm. You haven't been too involved with most of them but you are 
welcome to get involved. The critical things to remember is that you are 
0.01% of users and that in order to achieve wide scale acceptance we will 
need to increase the range of application development we can target beyond 
just simple dialogs. We also have a lot of questions to answer as we are 
blazing new trails, and inevitable compromises to make.
>
> > > Also notice that the "Starting up" texts from the dialog's
> > > initialization code do not work. Only the @debug works there!
> >
> > First of all there may be some problem with echo, but I need to
> > verify...
>
> As I said, it is not about echo itself. An echo is BTW a basic
> operation. If it doesn't work, there are big problems.

I'm not sure I can agree with that. Those echos can be done from a button. 
That is primarily where they would come from. In this case we're talking 
about running a dialog from the command line and expecting it to output to 
stdout prior to any user interaction. I'm trying to think of where this might 
apply to more than debug usage. First of all most Kommander dialogs will be 
run more like applications and they can output to the shell at any time. So 
at most it's a small problem, if that.
>
> > I see what you're saying. You would have to ask Michal to
> > really clear this up but I'll try. First is a bash call without a
> > shebang in a regular widget. IIRC we don't support this in non button
> > or script widgets.
>
> Why? This is inconsistent! Either require the sheebang or execBegin
> everywhere, or don't require it anywhere.

Okay... Now I remember one of the things Michal had related to me...
In a widget you are not executing a script by default. You're working with 
content. When you call an exec it proceeds to run as expected. In a button 
you are running an exec by default. It really wouldn't make sense otherwise. 
In this case we have the question of what to do, and what it will do is 
return a value from the exec, rather than run the exec in the shell as 
expected... After all, what is the point of escaping from the shell into the 
shell?

This is the kind of question that comes up. Looking at this I think possibly 
this might be cleared up with a paramater passing capability, but in fact the 
exec blocks are starting to piss me off. I am leaning toward eliminating them 
and requireing a call to a script object or Action, if for no other reason 
than to stop having this discussion.
>
> > For a while Michal had made it so that the buffer
> > was not flushed until closing.
>
> That's not the problem, as I said a completely unrelated echo "" fixes
> the bug.
>
> > I'm not sure about no echo. I believe
> > echo works in Kommander script. Anyway this is really a debug type
> > message. I know I can send text from a button to stdout. I added this
> > line to the close button and it works...
>
> Sure, it works some time. But not always. And printing debug information
> is not always possible with @debug, as it does not know about the shell
> variables.

Why don't you RTFM? "Sometimes" is not arbitrary here and passing shell 
variables... This has been discussed on the list before. If you want to 
compromise and use bash then you must decide to use bash everywhere, and 
remember that under some circumstances some things are impossible with bash. 
Fortunately for most people bash is to arcane to mess with. In the mean time 
we need to debug the functionality that the other 99.99% of users will be 
working with.
>
> > Actually it's pretty much the same as the others. All it really
> > offers is a slightly different notation for widgets with the same
> > functions, which is handled by the function browser, and the addition
> > of if/elseif/else/endif along with local variables.
>
> Still, it is a new language, and not something that extends any other
> language, which was the idea behind Kommander.

Let's put this to bed Andras. I couldn't communicate with my tax driver in 
Spain and I know more Spanish words then are in the "new language" here. 
Everything in it can be written by point and click. HTML has more elements 
than the structural flow, and they are so similar to bash, Javascript and 
other languages that it's pretty simple. What we need is to work on refining 
things, like a decent text editor.
>
> > basic operations. Complex programming tasks would be what you would
> > write in scripts with other languages, but ironically about 80% of
> > programming isn't that complex.
>
> Look, what I have to do here is:
> - I have a list of entries in a form of
> 1 filename
> 2 filename
> 3 filename
> - put the filenames in a list, and when you delete/download them, get
> back the number associated with the selected filenames.
> I don't find it complex. Bash has it's limitations to handle arrays, but
> Kommander has it's limitations to cooperate with bash when needed, and
> this caused my headache.

The limitations are NOT in Kommander. They are the problem with parallel 
execution and last I checked bash doesn't have a thread model and if it did 
then you would need a cluster with some serious bogomips to see output before 
you got sleepy.
>
> > > No, but why isn't the Function browser enough? Oh, well, it doesn't
> > > even tell me the possible return values.
> >
> > Is this really Andras? You obviously missed it in the function
> > browser... unless someone took it out.
>
> String.contains:
> Description: Checks if the the string contains the given substring.
> Syntax: contains(QString string, QString substring)
>
> But what does it return?

True or false. In the case of the @ functions it is 1 | 0.
>
> String.isEmpty, isNumber is the same. I see that others have it, but
> these are not, and it's enough if I just wanted to use one of these.
>
> ListBox.selection
> Description: Returns selected text or text of current item.
> Syntax: selection
>
> And what does it return if multiple items are selected?

I have to look and see. It seems to me there is another technique for this 
which is to use the widget selection which returns the selected elements in 
\n separated text. Then you can loop through that.
>
> I'm sure I could find others as well.

What is your point? If this was Quanta and you were me I think you'd be asking 
for a patch now.
>
> > I believe it is in the docs which regrettably are probably only on
> > site, but they are there. We really need to get an effort going to
> > fix all this for KDE 4.
>
> It is not mentioned in the "String Function Group" of the handbook.

Our handbook upkeep suffered during the time we were in a lot of quick change. 
It was also less than convenient that this all occurred where Coolo froze 
everything at once.
>
> > > Wrong.
> > > @forEach(i, @global(files))
> > >  @debug(@i)
> > > @endfor
> > >
> > > Just works.
> >
> > Really? Where's Michal?
>
> Who knows? ;-)
>
> > > But
> > >
> > > @forEach(i, @global(files))
> > >  @debug(@i)
> > > @# @if (@String.contains("@global(selection)", "@i")!="")
> > > @#    @debug(@i)
> > > @# @endif
> > > @endfor
> > >
> > > does not, and does not work even with @end instead of @endfor. See
> > > the commented out lines?
> > > Not talking about the fact that the function browser says one shoud
> > > use "@endif" for @foreach...
> >
> > Somebody should fix this, but I thought I worked on that at 3 am the
> > day before a string freeze. I can't believe this.
>
> Yeah, the documentation is easy to fix (and it qualifies for a bugfix,
> so I will do it, once I learn which is the correct way to close @if,
> @for and @forEach), but still, why doesn't the above work?

I don't know. I'll look at it later. I was spending a lot of time in this at 
one time and really my being busy and not testing as much didn't help Michal 
any with all the things going on. What we need to do is work on our teamwork 
here.
>
> > This would work by using a variable pool in Kommander and passing
> > parameters. Besides, if a program is fairly modular you don't need to
> > deal with more than a little integration of data.
>
> I told you about my little problem to solve. I don't need more than 2
> arrays and a little logic.

It should be easy. I'm happy to help but I can't actually test as my camera 
isn't the same. If you can recommend a similar shell operation we can work on 
that and you can translate it.
>
> > The language interaction question is pretty fundamental. You have two
> > programs separately executing and you need to synchronize between the
> > two at particular points to exchange data. This is just ugly.
>
> Yes, so you should make it one program, instead of two separate ones.

That's not possible without writing modules for every language you want to 
use. I'm sure you know that, and my concern is that such an endeavor could 
devolve into a lot of problems. Unlike you I'm not unhappy with the current 
solution because I expect DCOP enabled languages would work... I just don't 
have anyone testing it. If you have an opinion maybe we can solicit this, but 
I can tell you right now bash is not a prime candidate.
>
> > Attached is a dialog to try.
>
> This is not what I want and this one I could do it quite easily before
> as well. This is just widget manipulating without real content in the
> back.
>
> Andras
Didn't you just tell someone their description of a problem was inadequate for 
you to help? I'd really like to be able to help you.
-- 
Eric Laffoon
Project Lead - kdewebdev module