open-nomad/client/logmon/plugin.go

81 lines
1.7 KiB
Go
Raw Normal View History

package logmon
import (
"context"
Use init to handle plugin invocation Currently, nomad "plugin" processes (e.g. executor, logmon, docker_logger) are started as CLI commands to be handled by command CLI framework. Plugin launchers use `discover.NomadBinary()` to identify the binary and start it. This has few downsides: The trivial one is that when running tests, one must re-compile the nomad binary as the tests need to invoke the nomad executable to start plugin. This is frequently overlooked, resulting in puzzlement. The more significant issue with `executor` in particular is in relation to external driver: * Plugin must identify the path of invoking nomad binary, which is not trivial; `discvoer.NomadBinary()` now returns the path to the plugin rather than to nomad, preventing external drivers from launching executors. * The external driver may get a different version of executor than it expects (specially if we make a binary incompatible change in future). This commit addresses both downside by having the plugin invocation handling through an `init()` call, similar to how libcontainer init handler is done in [1] and recommened by libcontainer [2]. `init()` will be invoked and handled properly in tests and external drivers. For external drivers, this change will cause external drivers to launch the executor that's compiled against. There a are a couple of downsides to this approach: * These specific packages (i.e executor, logmon, and dockerlog) need to be careful in use of `init()`, package initializers. Must avoid having command execution rely on any other init in the package. I prefixed files with `z_` (golang processes files in lexical order), but ensured we don't depend on order. * The command handling is spread in multiple packages making it a bit less obvious how plugin starts are handled. [1] drivers/shared/executor/libcontainer_nsenter_linux.go [2] https://github.com/opencontainers/runc/tree/eb4aeed24ffbf8e2d740fafea39d91faa0ee84d0/libcontainer#using-libcontainer
2019-05-13 15:59:52 +00:00
"os"
"os/exec"
hclog "github.com/hashicorp/go-hclog"
plugin "github.com/hashicorp/go-plugin"
"github.com/hashicorp/nomad/client/logmon/proto"
"github.com/hashicorp/nomad/plugins/base"
"google.golang.org/grpc"
)
// LaunchLogMon launches a new logmon or reattaches to an existing one.
// TODO: Integrate with base plugin loader
func LaunchLogMon(logger hclog.Logger, reattachConfig *plugin.ReattachConfig) (LogMon, *plugin.Client, error) {
2019-01-05 00:08:47 +00:00
logger = logger.Named("logmon")
Use init to handle plugin invocation Currently, nomad "plugin" processes (e.g. executor, logmon, docker_logger) are started as CLI commands to be handled by command CLI framework. Plugin launchers use `discover.NomadBinary()` to identify the binary and start it. This has few downsides: The trivial one is that when running tests, one must re-compile the nomad binary as the tests need to invoke the nomad executable to start plugin. This is frequently overlooked, resulting in puzzlement. The more significant issue with `executor` in particular is in relation to external driver: * Plugin must identify the path of invoking nomad binary, which is not trivial; `discvoer.NomadBinary()` now returns the path to the plugin rather than to nomad, preventing external drivers from launching executors. * The external driver may get a different version of executor than it expects (specially if we make a binary incompatible change in future). This commit addresses both downside by having the plugin invocation handling through an `init()` call, similar to how libcontainer init handler is done in [1] and recommened by libcontainer [2]. `init()` will be invoked and handled properly in tests and external drivers. For external drivers, this change will cause external drivers to launch the executor that's compiled against. There a are a couple of downsides to this approach: * These specific packages (i.e executor, logmon, and dockerlog) need to be careful in use of `init()`, package initializers. Must avoid having command execution rely on any other init in the package. I prefixed files with `z_` (golang processes files in lexical order), but ensured we don't depend on order. * The command handling is spread in multiple packages making it a bit less obvious how plugin starts are handled. [1] drivers/shared/executor/libcontainer_nsenter_linux.go [2] https://github.com/opencontainers/runc/tree/eb4aeed24ffbf8e2d740fafea39d91faa0ee84d0/libcontainer#using-libcontainer
2019-05-13 15:59:52 +00:00
bin, err := os.Executable()
if err != nil {
return nil, nil, err
}
conf := &plugin.ClientConfig{
HandshakeConfig: base.Handshake,
Plugins: map[string]plugin.Plugin{
2019-01-05 00:08:47 +00:00
"logmon": &Plugin{},
},
AllowedProtocols: []plugin.Protocol{
plugin.ProtocolGRPC,
},
2019-01-05 00:08:47 +00:00
Logger: logger,
}
// Only set one of Cmd or Reattach
if reattachConfig == nil {
conf.Cmd = exec.Command(bin, "logmon")
} else {
conf.Reattach = reattachConfig
}
client := plugin.NewClient(conf)
rpcClient, err := client.Client()
if err != nil {
return nil, nil, err
}
raw, err := rpcClient.Dispense("logmon")
if err != nil {
return nil, nil, err
}
l := raw.(LogMon)
return l, client, nil
}
type Plugin struct {
plugin.NetRPCUnsupportedPlugin
impl LogMon
}
func NewPlugin(i LogMon) plugin.Plugin {
return &Plugin{impl: i}
}
func (p *Plugin) GRPCServer(broker *plugin.GRPCBroker, s *grpc.Server) error {
proto.RegisterLogMonServer(s, &logmonServer{
impl: p.impl,
broker: broker,
})
return nil
}
func (p *Plugin) GRPCClient(ctx context.Context, broker *plugin.GRPCBroker, c *grpc.ClientConn) (interface{}, error) {
return &logmonClient{
doneCtx: ctx,
client: proto.NewLogMonClient(c),
}, nil
}